MongoDB集群部署——Replica Set

Roy
数据库

mongodb的集群部署有3种方式，这里记录我在使用Replica Set（副本集）部署时的步骤。

首先这种方式至少需要3个节点——主、备、仲裁节点。主备节点存储数据，仲裁节点不存储数据，负责决定主节点挂掉之后哪个备节点升级为主节点。客户端同时连接主节点与备节点，不连接仲裁节点。

安装mongodb，CentOS下安装rpmfusion源后使用yum安装即可:yum install mongodb mongodb-server

安装完成后，修改/etc/mongodb.conf 这个配置文件。

bind_ip = 192.168.4.50
port = 27017
fork = true
pidfilepath = /var/run/mongodb/mongodb.pid
logpath = /var/log/mongodb/mongodb.log
dbpath =/var/lib/mongodb
journal = true
directoryperdb=true
logappend=true
replSet=testrs
noprealloc=true

其中注意修改bind_ip为各个节点的IP，下面的4行是我添加的：

  1. directoryperdb：为每一个数据库按照数据库名建立文件夹存放
  2. logappend：以追加的方式记录日志
  3. replSet：replica set的名字
  4. noprealloc：不预先分配存储

修改完成后，使用service mongod start即可启动服务。

随便链接一个节点后，进行配置：

[root@pxc_bj63 ~]# service mongod start
Starting mongod:                                           [  OK  ]
[root@pxc_bj63 ~]# clear
[root@pxc_bj63 ~]# mongo 192.168.4.63:27017
MongoDB shell version: 2.4.12
connecting to: 192.168.4.63:27017/test
> use admin
switched to db admin
> conf = {_id:"testrs",members:[{_id:0,host:"192.168.4.63:27017",priority:2},{_id:1,host:"192.168.4.64:27017",priority:1},{_id:2,host:"192.168.4.50:27017",arbiterOnly:true}]};
{
	"_id" : "testrs",
	"members" : [
		{
			"_id" : 0,
			"host" : "192.168.4.63:27017",
			"priority" : 2
		},
		{
			"_id" : 1,
			"host" : "192.168.4.64:27017",
			"priority" : 1
		},
		{
			"_id" : 2,
			"host" : "192.168.4.50:27017",
			"arbiterOnly" : true
		}
	]
}

注意最初的_id的值就是我们配置文件中设定的replSet的值，arbiterOnly是大写的O，配置完成后进行初始化：

> rs.initiate(conf)
{
	"info" : "Config now saved locally.  Should come online in about a minute.",
	"ok" : 1
}
> rs.status()
{
	"set" : "testrs",
	"date" : ISODate("2015-03-03T03:17:25Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.4.63:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 460,
			"optime" : Timestamp(1425352584, 1),
			"optimeDate" : ISODate("2015-03-03T03:16:24Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "192.168.4.64:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 60,
			"optime" : Timestamp(1425352584, 1),
			"optimeDate" : ISODate("2015-03-03T03:16:24Z"),
			"lastHeartbeat" : ISODate("2015-03-03T03:17:25Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:17:25Z"),
			"pingMs" : 0,
			"syncingTo" : "192.168.4.63:27017"
		},
		{
			"_id" : 2,
			"name" : "192.168.4.50:27017",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 58,
			"lastHeartbeat" : ISODate("2015-03-03T03:17:25Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:17:24Z"),
			"pingMs" : 1
		}
	],
	"ok" : 1
}
testrs:PRIMARY>

插入一条数据试试：

testrs:PRIMARY> use test
switched to db test
testrs:PRIMARY> db.testdb.insert({"test1":"i'm master"})

目前，备节点是无法查询数据的：

[root@pxc_bj_64 ~]# mongo 192.168.4.64
MongoDB shell version: 2.4.12
connecting to: 192.168.4.64/test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	http://docs.mongodb.org/
Questions? Try the support group
	http://groups.google.com/group/mongodb-user
testrs:SECONDARY> use test;
switched to db test
testrs:SECONDARY> show tables;
Tue Mar  3 11:41:00.274 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:128

需要进行配置：

testrs:SECONDARY> db.getMongo().setSlaveOk();
testrs:SECONDARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }

设定setSlaveOK后便可以进行数据查询了。

现在，停止主库的服务，再查看整个集群的状态

testrs:SECONDARY> rs.status();
{
	"set" : "testrs",
	"date" : ISODate("2015-03-03T03:48:40Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.4.63:27017",
			"health" : 0,
			"state" : 8,
			"stateStr" : "(not reachable/healthy)",
			"uptime" : 0,
			"optime" : Timestamp(1425353872, 1),
			"optimeDate" : ISODate("2015-03-03T03:37:52Z"),
			"lastHeartbeat" : ISODate("2015-03-03T03:48:39Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:47:39Z"),
			"pingMs" : 0
		},
		{
			"_id" : 1,
			"name" : "192.168.4.64:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 2317,
			"optime" : Timestamp(1425353872, 1),
			"optimeDate" : ISODate("2015-03-03T03:37:52Z"),
			"self" : true
		},
		{
			"_id" : 2,
			"name" : "192.168.4.50:27017",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 1905,
			"lastHeartbeat" : ISODate("2015-03-03T03:48:39Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:48:40Z"),
			"pingMs" : 1
		}
	],
	"ok" : 1
}

此时原来的备节点变成了主节点，此时我们新插入一条数据：

testrs:PRIMARY> db.testdb.insert({"test2":"i'm new master"})
testrs:PRIMARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }
{ "_id" : ObjectId("54f52f70abb951d25573981e"), "test2" : "i'm new master" }

重启原来的主节点并查看状态：

[root@pxc_bj63 ~]# mongo 192.168.4.63
MongoDB shell version: 2.4.12
connecting to: 192.168.4.63/test
testrs:PRIMARY> rs.status()
{
	"set" : "testrs",
	"date" : ISODate("2015-03-03T03:50:57Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.4.63:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 59,
			"optime" : Timestamp(1425354608, 1),
			"optimeDate" : ISODate("2015-03-03T03:50:08Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "192.168.4.64:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 59,
			"optime" : Timestamp(1425354608, 1),
			"optimeDate" : ISODate("2015-03-03T03:50:08Z"),
			"lastHeartbeat" : ISODate("2015-03-03T03:50:56Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:50:57Z"),
			"pingMs" : 0,
			"lastHeartbeatMessage" : "syncing to: 192.168.4.63:27017",
			"syncingTo" : "192.168.4.63:27017"
		},
		{
			"_id" : 2,
			"name" : "192.168.4.50:27017",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 57,
			"lastHeartbeat" : ISODate("2015-03-03T03:50:56Z"),
			"lastHeartbeatRecv" : ISODate("2015-03-03T03:50:56Z"),
			"pingMs" : 0
		}
	],
	"ok" : 1
}

查询数据也同步过来了：

testrs:PRIMARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }
{ "_id" : ObjectId("54f52f70abb951d25573981e"), "test2" : "i'm new master" }

当主节点恢复后，备节点查询数据时会提示：

testrs:PRIMARY> db.testdb.find();
Tue Mar  3 11:53:00.354 DBClientCursor::init call() failed
Error: error doing query: failed
Tue Mar  3 11:53:00.355 trying reconnect to 192.168.4.64:27017
Tue Mar  3 11:53:00.356 reconnect 192.168.4.64:27017 ok
testrs:SECONDARY>

如果要向集群中新增节点，则进行如下操作：

cfg = rs.conf()
cfg.members[3] = {_id:3,host:"192.168.4.42:27017",priority:0.5}
rs.reconfig(cfg)

新的备份节点配置之前是无法读数据的，记得设定setSlaveOK，不过虽然可读但备份节点还是不可写的。如果想修改配置也同理，首先得到当前配置信息，以修改优先级为例：

cfg = rs.conf()
cfg.members[0].priority = 0.5
cfg.members[1].priority = 2
cfg.members[2].priority = 2
rs.reconfig(cfg)

一定记得reconfig后新配置才会生效。