最详细的 RocketMQ 集群部署指南,附完整脚本 + 实战配置(建议收藏)

RocketMQ 是使用 Java 开发的高性能的消息队列系统。它的部署方式和 Redis 不同,它直接使用我们熟悉的 jar 包的方式部署。从官网下载发布版二进制包(maven 编译后的 jar 包、启动脚本集合),进行安装。

单个 RocketMQ 共有两个进程:nameserver 和 broker , nameserver 相当于注册中心,用于管理集群的服务注册、发现、负载等功能,broker 负责消息的实际存储和推送。

接下我们就开始安装 RocketMQ,使用源码的方式进行安装,分为单机部署和集群部署。 本次安装使用的版本 RocketMQ v4.9.7,生产环境推荐集群部署(下文有介绍)。

本文根据我的实际开发经验,分享了 RocketMQ 部署方案和完整的执行脚本,以下所有的部署脚本都是经过实际生产环境验证过的,读者可以直接拿过来就使用,建议收藏~

单机部署

1、下载二进制包
1cd /home/recharge/local
2wget https://archive.apache.org/dist/rocketmq/4.9.7/rocketmq-all-4.9.7-bin-release.zip
3# 如果没有安装unzip,需要提前安装下:yum install -y unzip
4unzip rocketmq-all-4.9.7-bin-release.zip
5mv rocketmq-all-4.9.7-bin-release rocketmq-4.9.7
2、初始化配置文件
1mkdir -p ./rocketmq-namesrv-1/logs \
2./rocketmq-namesrv-1/store \
3./rocketmq-master-1/logs \
4./rocketmq-master-1/store 
 1vim ./rocketmq-namesrv-1/namesrv.properties
 2
 3# 编辑以下内容
 4
 5# NameServer 存储路径
 6storePathRootDir=/home/recharge/local/rocketmq-namesrv-1/store/
 7# NameServer 监听端口(默认为 9876)
 8listenPort=9876
 9# Broker 心跳超时时间(毫秒)
10waitTimeMillsInHeartbeat=30000
 1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-1/broker.conf 
 2vim ./rocketmq-master-1/broker.conf
 3
 4# 编辑以下内容
 5brokerClusterName = DefaultCluster
 6brokerName = broker-a
 7brokerId = 0
 8#表示几点做消息删除动作,默认是凌晨4点
 9deleteWhen = 04
10#在磁盘上保留消息的时长,单位是小时
11fileReservedTime = 48
12#有三个值:SYNC_MASTER,ASYNC_MASTER,SLAVE;
13brokerRole = ASYNC_MASTER
14#ASYNC_FLUSH异步刷盘,SYNC_FLUSH表示同步刷盘;SYNC_FLUSH消息写入磁盘后才返回成功状态,ASYNC_FLUSH不需要;
15flushDiskType = ASYNC_FLUSH
16
17#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
18brokerIP1=192.168.110.10
19
20namesrvAddr=127.0.0.1:9876
21
22listenPort=10911
23storePathRootDir=/home/recharge/local/rocketmq-master-1/store
24storePathCommitLog=/home/recharge/local/rocketmq-master-1/store/commitlog
25storePathConsumerQueue=/home/recharge/local/rocketmq-master-1/store/consumequeue
26storePathIndex=/home/recharge/local/rocketmq-master-1/store/index
27
28autoCreateTopicEnable=true
3、运行服务
 1#运行nameserver
 2nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-1/namesrv.properties &> ./rocketmq-namesrv-1/logs/namesrv.log &
 3tail -fn 100 ./rocketmq-namesrv-1/logs/namesrv.log
 4#输出:The Name Server boot success. serializeType=JSON
 5
 6#运行broker 之前如果在测试环境,内存不足时,需要更新JVM 启动参数
 7vim ./rocketmq-4.9.7/bin/runbroker.sh
 8# JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g"
 9# 改为:JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"
10
11# 运行broker
12nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-1/broker.conf &> ./rocketmq-master-1/logs/broker.log &
13tail -fn 100 ./rocketmq-master-1/logs/broker.log
14# 输出:The broker[broker-a, 192.168.0.120:10911] boot success. serializeType=JSON
15
16#检查进程
17ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
18ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
19
20#关闭服务
21sh ./rocketmq-4.9.7/bin/mqshutdown broker
22sh ./rocketmq-4.9.7/bin/mqshutdown namesrv

跟着之前的我分享的文章教程 Java 客户端连接 RocketMQ ,就可以开始本地使用啦。

集群部署

单机部署一般用于本地或测试,生产环境不推荐使用,原因有两点:

1、broker服务器机械故障会导致该节点消息永久丢失,无法恢复,采用1主1从节同步数据;

2、broker单节点故障会导致服务不可用,采用多个主节点部署方式;

3、nameserver 单节点故障导致网络连接异常,采用多个节点部署(注册中心是无状态的)。

多节点存储方案

nameserver 角色

监听端口

存储路径

server-1

9876

/home/recharge/local/rocketmq-namesrv-1/store/

server-1

9877

/home/recharge/local/rocketmq-namesrv-2/store/

server-3

9878

/home/recharge/local/rocketmq-namesrv-3/store/

Broker 节点

Broker 角色

brokerNamebrokerId

监听端口

存储路径

Master-1

broker-a

0

10911

/home/recharge/local/rocketmq-master-1/store

Slave-1

broker-a

1

11911

/home/recharge/local/rocketmq-slave-1/store

Master-2

broker-b

0

10912

/home/recharge/local/rocketmq-master-2/store

Slave-2

broker-b

1

11912

/home/recharge/local/rocketmq-slave-1/store

首先参考单机部署方案,将二进制下载到本地。部署方案将分为两步:安装 nameserver 集群 和 安装 Master-Slave 集群

安装 nameserver 集群

1、初始化文件目录

1cd /home/recharge/local
2
3mkdir -p ./rocketmq-namesrv-1/logs \
4./rocketmq-namesrv-1/store \
5./rocketmq-namesrv-2/logs \
6./rocketmq-namesrv-2/store \
7./rocketmq-namesrv-3/logs \
8./rocketmq-namesrv-3/store 

2、初始化 namesrver 配置文件

1vim ./rocketmq-namesrv-1/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-1/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9876
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000
1vim ./rocketmq-namesrv-2/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-2/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9877
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000
1vim ./rocketmq-namesrv-3/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-3/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9878
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000

3、运行 nameserver 集群

 1#运行nameserver1
 2nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-1/namesrv.properties &> ./rocketmq-namesrv-1/logs/namesrv.log &
 3tail -fn 100 ./rocketmq-namesrv-1/logs/namesrv.log
 4#输出:The Name Server boot success. serializeType=JSON
 5ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
 6
 7#运行nameserver2
 8nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-2/namesrv.properties &> ./rocketmq-namesrv-2/logs/namesrv.log &
 9tail -fn 100 ./rocketmq-namesrv-2/logs/namesrv.log
10#输出:The Name Server boot success. serializeType=JSON
11ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
12
13#运行nameserver3
14nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-3/namesrv.properties &> ./rocketmq-namesrv-3/logs/namesrv.log &
15tail -fn 100 ./rocketmq-namesrv-3/logs/namesrv.log
16#输出:The Name Server boot success. serializeType=JSON
17ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup

4、关闭服务

1sh ./rocketmq-4.9.7/bin/mqshutdown namesrv
安装Master-Slave集群

1、初始化文件目录

 1cd /home/recharge/local
 2
 3mkdir -p ./rocketmq-master-1/logs \
 4./rocketmq-master-1/store \
 5./rocketmq-slave-1/logs \
 6./rocketmq-slave-1/store 
 7
 8mkdir -p ./rocketmq-master-2/logs \
 9./rocketmq-master-2/store \
10./rocketmq-slave-2/logs \
11./rocketmq-slave-2/store  

2、初始化master-slave配置文件

 1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-1/broker.conf 
 2vim ./rocketmq-master-1/broker.conf
 3
 4# 编辑以下内容
 5brokerClusterName=rocketmq-cluster
 6brokerName=broker-a
 7brokerId=0
 8deleteWhen=04
 9fileReservedTime=48
10brokerRole=ASYNC_MASTER
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16#Broker 外部服务端口(默认 10911) → 用于处理消息存储和传输
17listenPort=10911
18#Broker 高速存储同步端口(默认 10912) → 用于 Master-Slave 复制
19haListenPort=10912
20#Broker Web 控制台端口(默认 10909)
21dLegerPort=10909
22
23storePathRootDir=/home/recharge/local/rocketmq-master-1/store
24storePathCommitLog=/home/recharge/local/rocketmq-master-1/store/commitlog
25storePathConsumerQueue=/home/recharge/local/rocketmq-master-1/store/consumequeue
26storePathIndex=/home/recharge/local/rocketmq-master-1/store/index
27storePathConsumerOffset=/home/recharge/local/rocketmq-master-1/store/consumerOffset
28storePathConsumerOffset=/home/recharge/local/rocketmq-master-1/store/consumerOffset
29
30autoCreateTopicEnable=true
 1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-slave-1/broker.conf 
 2vim ./rocketmq-slave-1/broker.conf
 3
 4# 编辑以下内容
 5brokerClusterName=rocketmq-cluster
 6brokerName=broker-a
 7brokerId=1
 8deleteWhen=04
 9fileReservedTime=48
10brokerRole=SLAVE
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16listenPort=20911
17haListenPort=20912
18dLegerPort=20909
19
20storePathRootDir=/home/recharge/local/rocketmq-slave-1/store
21storePathCommitLog=/home/recharge/local/rocketmq-slave-1/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-slave-1/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-slave-1/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-slave-1/store/consumerOffset
25
26autoCreateTopicEnable=false
 1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-2/broker.conf 
 2vim ./rocketmq-master-2/broker.conf
 3
 4# 编辑以下内容
 5brokerClusterName=rocketmq-cluster
 6brokerName=broker-b
 7brokerId=0
 8deleteWhen=04
 9fileReservedTime=48
10brokerRole=ASYNC_MASTER
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16listenPort=11911
17haListenPort=11912
18dLegerPort=11909
19
20storePathRootDir=/home/recharge/local/rocketmq-master-2/store
21storePathCommitLog=/home/recharge/local/rocketmq-master-2/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-master-2/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-master-2/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-master-2/store/consumerOffset
25
26autoCreateTopicEnable=true
 1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-slave-2/broker.conf 
 2vim ./rocketmq-slave-2/broker.conf
 3
 4# 编辑以下内容
 5brokerClusterName=rocketmq-cluster
 6brokerName=broker-b
 7brokerId=1
 8deleteWhen=04
 9fileReservedTime=48
10brokerRole=SLAVE
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13listenPort=21911
14haListenPort=21912
15dLegerPort=21909
16#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
17brokerIP1=192.168.110.10
18
19
20storePathRootDir=/home/recharge/local/rocketmq-slave-2/store
21storePathCommitLog=/home/recharge/local/rocketmq-slave-2/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-slave-2/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-slave-2/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-slave-2/store/consumerOffset
25
26autoCreateTopicEnable=false

3、运行服务

 1# 运行broker-a-0
 2nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-1/broker.conf &> ./rocketmq-master-1/logs/broker.log &
 3tail -fn 100 ./rocketmq-master-1/logs/broker.log
 4ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
 5
 6# 运行broker-a-1
 7nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-slave-1/broker.conf &> ./rocketmq-slave-1/logs/broker.log &
 8tail -fn 100 ./rocketmq-slave-1/logs/broker.log
 9ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
10
11
12# 运行broker-b-0
13nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-2/broker.conf &> ./rocketmq-master-2/logs/broker.log &
14tail -fn 100 ./rocketmq-master-2/logs/broker.log
15ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
16
17
18# 运行broker-b-1
19nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-slave-2/broker.conf &> ./rocketmq-slave-2/logs/broker.log &
20tail -fn 100 ./rocketmq-slave-2/logs/broker.log
21ps -aux|grep org.apache.rocketmq.broker.BrokerStartup

4、关闭服务

1sh ./rocketmq-4.9.7/bin/mqshutdown broker

至此,RocketMQ 集群环境部署完成,生产环境就可以投入使用~

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值