0x00 前言
mongodb是一款基于分布式文件存储的数据库，具有高性能、可扩展、易部署、易使用等特点。官方也提供了丰富的命令行工具来操作。
0x01 部署mongodb服务
部署mongodb服务可以直接使用docker镜像：
$ docker search mongoNAME DESCRIPTION STARS OFFICIAL AUTOMATEDmongo MongoDB document databases provide high avai… 6656 [OK]mongo-express Web-based MongoDB admin interface, written w… 640 [OK]tutum/mongodb MongoDB Docker image – listens in port 27017… 230 [OK]bitnami/mongodb Bitnami MongoDB Docker Image 109 [OK]mongoclient/mongoclient Official docker image for Mongoclient, featu… 79 [OK]mongooseim/mongooseim Small docker image for MongooseIM - robust a… 19frodenas/mongodb A Docker Image for MongoDB 18 [OK]cvallance/mongo-k8s-sidecar Kubernetes side car to setup and maintain a … 14 [OK]centos/mongodb-32-centos7 MongoDB NoSQL database server 8circleci/mongo CircleCI images for MongoDB 8 [OK]arm64v8/mongo MongoDB document databases provide high avai… 7istepanov/mongodump Docker image with mongodump running as a cro… 6 [OK]centos/mongodb-36-centos7 MongoDB NoSQL database server 5eses/mongodb_exporter mongodb exporter for prometheus 5 [OK]centos/mongodb-26-centos7 MongoDB NoSQL database server 5webhippie/mongodb Docker images for MongoDB 4 [OK]requilence/mongodb-backup mongo backup container 4 [OK]centos/mongodb-34-centos7 MongoDB NoSQL database server 3neowaylabs/mongodb-mms-agent This Docker image with MongoDB Monitoring Ag… 3 [OK]ansibleplaybookbundle/mongodb-apb An APB to deploy MongoDB. 1 [OK]ekesken/mongo docker image for mongo that is configurable … 1 [OK]andreasleicher/mongo-azure-backup a docker container to backup a mongodb using… 1 [OK]openshift/mongodb-24-centos7 DEPRECATED: A Centos7 based MongoDB v2.4 ima… 1ccitest/mongo CircleCI test images for Mongo 0 [OK]targetprocess/mongodb_exporter MongoDB exporter for prometheus 0 [OK]$ docker pull mongo:latest$ docker run -itd --name mongo -p 27017:27017 mongo --auth
此时，mongodb服务已经正常运行起来了，还需要配置下账号。
使用docker exec -it mongo bash命令进入到shell环境后，输入mongo命令进入交互式命令行：
> use adminswitched to db admin> db.createUser({ user: "root", pwd: "root", roles: [{ role: "root", db: "admin" }] })Successfully added user: {"user" : "root","roles" : [{"role" : "root","db" : "admin"}]}COPY
权限角色说明：
Read：允许用户读取指定数据库
readWrite：允许用户读写指定数据库
dbAdmin：允许用户在指定数据库中执行管理函数，如索引创建、删除，查看统计或访问system.profile
userAdmin：允许用户向system.users集合写入，可以找指定数据库里创建、删除和管理用户
clusterAdmin：只在admin数据库中可用，赋予用户所有分片和复制集相关函数的管理权限
readAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读权限
readWriteAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读写权限
userAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的userAdmin权限
dbAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的dbAdmin权限
root：只在admin数据库中可用。超级账号，超级权限
0x02 安装mongo客户端
$ apt install mongodb-clientsCOPY
进入mongo shell：
$ mongo ${mongo_server}:${mongo_port} -u ${username} -p ${password} --authenticationDatabase adminCOPY
mongodb中分为不同的数据库，默认数据库为test，使用use db切换数据库时，如果数据库不存在会自动创建。
0x03 常用mongo命令
> show dbsadmin 0.000GBconfig 0.000GBlocal 0.000GB> use testswitched to db test> dbtest> db.table.insert({"title":"drunkdream", "body": "Hello world"})WriteResult({ "nInserted" : 1 })> db.table.count()1> db.table.dataSize()66> db.table.insert({"title":"drunkdream123", "body": "Test"})WriteResult({ "nInserted" : 1 })> db.table.find().skip(1).limit(1){ "_id" : ObjectId("5e706e4651e5f0d32c1eb3e0"), "title" : "drunkdream123", "body" : "Test" }> db.table.find({title: /^drunkdream/}){ "_id" : ObjectId("5e706a7051e5f0d32c1eb3df"), "title" : "drunkdream", "body" : "Hello world" }{ "_id" : ObjectId("5e706e4651e5f0d32c1eb3e0"), "title" : "drunkdream123", "body" : "Test" }> db.getCollectionNames()[ "table" ]> db.printCollectionStats()table{"ns" : "test.table","size" : 66,"count" : 1,"avgObjSize" : 66,"storageSize" : 20480,"capped" : false,...}> db.stats(){"db" : "test","collections" : 1,"views" : 0,"objects" : 1,"avgObjSize" : 66,"dataSize" : 66,"storageSize" : 4096,"numExtents" : 0,"indexes" : 1,"indexSize" : 4096,"scaleFactor" : 1,"fsUsedSize" : 34517733376,"fsTotalSize" : 109059317760,"ok" : 1}
find命令支持的查询条件：
$gt: >
$lt: <
$gte: >=
$lte: <=
$ne: !=
例如：{age: {$gte: 23, $lte: 26}}
0x04 其它mongo命令
db.table.drop()- 删除集合db.dropDatabase()- 删除当前数据库db.repairDatabase()- 修复数据库db.getPrevError()- 查询之前的错误信息
0x05 迁移数据
mongodump
$ mongodump -h ${mongo_server} --port ${mongo_port} -d ${db_name} -o ${save_dir} -u ${username} -p ${password} --authenticationDatabase adminCOPY
该命令可以将整个数据库dump到本地，如果只需要dump其中的一个集合，可以使用以下参数：--collection ${coll_name}。
mongorestore
$ mongorestore -h ${mongo_server} --port ${mongo_port} -d ${db_name} ${save_dir}/${db_name} -u ${username} -p ${password} --authenticationDatabase adminCOPY
该命令可以将备份下来的数据库还原到目标数据库中，修改-d参数可以修改目标数据库的名称。
如果目标数据库存在，需要增加--drop参数进行还原，避免报错。
迁移脚本
写了个简单的迁移脚本：
# -*- coding: utf-8 -*-import argparseimport osimport tempfileimport timedef sync_mongo(src_host, src_port, dst_host, dst_port, db, collection=None, username='root', password='root', drop=False):time0 = time.time()backup_dir = tempfile.mkdtemp('.backup').split(os.path.sep)[-1]auth_cmdline = ' -u %s -p %s --authenticationDatabase admin' % (username, password)cmdline = 'mongodump -h %s --port=%d -d %s -o %s' % (src_host, src_port, db, backup_dir)if collection:cmdline += ' --collection %s' % collectioncmdline += auth_cmdlineprint(cmdline)os.system(cmdline)time1 = time.time()print('backup %s from %s complete, cost %fs\n' % (db, src_host, time1 - time0))cmdline = 'mongorestore -h %s --port=%d -d %s %s/%s' % (dst_host, dst_port, db, backup_dir, db)if drop:cmdline += ' --drop'cmdline += auth_cmdlineprint(cmdline)os.system(cmdline)time1 = time.time()print('sync data from %s/%s to %s/%s complete, total cost %fs' % (src_host, db, dst_host, db, time1 - time0))if __name__ == '__main__':parser = argparse.ArgumentParser(description='Mongodb sync data script.')parser.add_argument('src_host', help='source host')parser.add_argument('dst_host', help='destination host')parser.add_argument('db_name', help='database name')parser.add_argument('--src-port', type=int, help='source port', default=27017)parser.add_argument('--dst-port', type=int, help='destination port', default=27017)parser.add_argument('--collection', help='collection to sync')parser.add_argument('--drop', help='drop collection before restore', default=False, action='store_true')args = parser.parse_args()sync_mongo(args.src_host, args.src_port, args.dst_host, args.dst_port, args.db_name, args.collection, drop=args.drop)
增量备份
oplog是local数据库中的一个固定集合，Secondary就是通过查看Primary的oplog这个集合来进行复制的。每个节点都有oplog，记录这从主节点复制过来的信息，这样每个成员都可以作为同步源给其他节点。
FROM mongo:3.6.3RUN printf "mongod --replSet rs0 &\nsleep 2\nmongo local --eval \"rs.initiate()\"\nmongo local --eval \"rs.conf()\"\nsleep infinity" > /usr/local/bin/init-mongo.shENTRYPOINT "sh /usr/local/bin/init-mongo.sh"
可以使用上面的Dockerfile生成一个自动开启oplog的容器。
rs0:PRIMARY> use localswitched to db localrs0:PRIMARY> db.getCollectionNames()["me","oplog.rs","replset.election","replset.minvalid","startup_log","system.replset","system.rollback.id"]rs0:PRIMARY> db.oplog.rs.find(){ "ts" : Timestamp(1584972071, 1), "h" : NumberLong("7155205047322550757"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:01:11.490Z"), "o" : { "msg" : "initiating set" } }{ "ts" : Timestamp(1584972073, 1), "t" : NumberLong(1), "h" : NumberLong("3098310879886849971"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:01:13.536Z"), "o" : { "msg" : "new primary" } }{ "ts" : Timestamp(1584972073, 2), "t" : NumberLong(1), "h" : NumberLong("7423790971296558722"), "v" : 2, "op" : "c", "ns" : "config.$cmd", "ui" : UUID("ac77fd00-b096-4062-b7ca-ac7f187c1fc7"), "wall" : ISODate("2020-03-23T14:01:13.576Z"), "o" : { "create" : "transactions", "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "config.transactions" } } }{ "ts" : Timestamp(1584972073, 3), "t" : NumberLong(1), "h" : NumberLong("-2051859930978240465"), "v" : 2, "op" : "c", "ns" : "admin.$cmd", "ui" : UUID("4a0104be-5d4f-49db-9204-d778d83c64dd"), "wall" : ISODate("2020-03-23T14:01:13.606Z"), "o" : { "create" : "system.keys", "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "admin.system.keys" } } }{ "ts" : Timestamp(1584972073, 4), "t" : NumberLong(1), "h" : NumberLong("-4130342990996856185"), "v" : 2, "op" : "i", "ns" : "admin.system.keys", "ui" : UUID("4a0104be-5d4f-49db-9204-d778d83c64dd"), "wall" : ISODate("2020-03-23T14:01:13.606Z"), "o" : { "_id" : NumberLong("6807403218608324609"), "purpose" : "HMAC", "key" : BinData(0,"h/kcD++ijF/AUzwUqqXHTKG4Jwk="), "expiresAt" : Timestamp(1592748073, 0) } }{ "ts" : Timestamp(1584972073, 5), "t" : NumberLong(1), "h" : NumberLong("3814221418240704217"), "v" : 2, "op" : "i", "ns" : "admin.system.keys", "ui" : UUID("4a0104be-5d4f-49db-9204-d778d83c64dd"), "wall" : ISODate("2020-03-23T14:01:13.606Z"), "o" : { "_id" : NumberLong("6807403218608324610"), "purpose" : "HMAC", "key" : BinData(0,"4Ds1uk/bTNYytP0hMkXX9ilgz38="), "expiresAt" : Timestamp(1600524073, 0) } }{ "ts" : Timestamp(1584972093, 1), "t" : NumberLong(1), "h" : NumberLong("7954269847273383577"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:01:33.576Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972103, 1), "t" : NumberLong(1), "h" : NumberLong("1461838052525134608"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:01:43.576Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972113, 1), "t" : NumberLong(1), "h" : NumberLong("-2878317090536866771"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:01:53.576Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972123, 1), "t" : NumberLong(1), "h" : NumberLong("8386919002466516906"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:03.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972133, 1), "t" : NumberLong(1), "h" : NumberLong("3752867656297542596"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:13.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972143, 1), "t" : NumberLong(1), "h" : NumberLong("-6914327186052916342"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:23.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972153, 1), "t" : NumberLong(1), "h" : NumberLong("4905151581311694538"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:33.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972163, 1), "t" : NumberLong(1), "h" : NumberLong("2195348018262890539"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:43.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972173, 1), "t" : NumberLong(1), "h" : NumberLong("-945081611143254792"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:02:53.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972183, 1), "t" : NumberLong(1), "h" : NumberLong("-8550993957527781683"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:03:03.577Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972193, 1), "t" : NumberLong(1), "h" : NumberLong("2027336719720729041"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:03:13.578Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972203, 1), "t" : NumberLong(1), "h" : NumberLong("8383046467552376073"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:03:23.578Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972213, 1), "t" : NumberLong(1), "h" : NumberLong("-2718017147947765099"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:03:33.578Z"), "o" : { "msg" : "periodic noop" } }{ "ts" : Timestamp(1584972223, 1), "t" : NumberLong(1), "h" : NumberLong("4265940161146074795"), "v" : 2, "op" : "n", "ns" : "", "wall" : ISODate("2020-03-23T14:03:43.578Z"), "o" : { "msg" : "periodic noop" } }Type "it" for moreCOPY
可以看到，oplog.rs集合里已经有数据了。
备份数据命令：
$ mongodump -h ${mongo_server} --port ${mongo_port} -o ${save_dir} -u ${username} -p ${password} --authenticationDatabase admin --oplogCOPY
使用
--oplog参数时不能指定数据库，只能全量备份
还原数据命令：
$ mongorestore -h ${mongo_server} --port ${mongo_port} -u ${username} -p ${password} --authenticationDatabase admin --oplogReplay --dir ${save_dir}COPY
Be the first guy leaving a comment!