RocketMQ 是使用 Java 开发的高性能的消息队列系统。它的部署方式和 Redis 不同,它直接使用我们熟悉的 jar 包的方式部署。从官网下载发布版二进制包(maven 编译后的 jar 包、启动脚本集合),进行安装。
单个 RocketMQ 共有两个进程:nameserver 和 broker , nameserver 相当于注册中心,用于管理集群的服务注册、发现、负载等功能,broker 负责消息的实际存储和推送。
接下我们就开始安装 RocketMQ,使用源码的方式进行安装,分为单机部署和集群部署。 本次安装使用的版本 RocketMQ v4.9.7,生产环境推荐集群部署(下文有介绍)。
本文根据我的实际开发经验,分享了 RocketMQ 部署方案和完整的执行脚本,以下所有的部署脚本都是经过实际生产环境验证过的,读者可以直接拿过来就使用,建议收藏~
单机部署
1、下载二进制包
1cd /home/recharge/local
2wget https://archive.apache.org/dist/rocketmq/4.9.7/rocketmq-all-4.9.7-bin-release.zip
3# 如果没有安装unzip,需要提前安装下:yum install -y unzip
4unzip rocketmq-all-4.9.7-bin-release.zip
5mv rocketmq-all-4.9.7-bin-release rocketmq-4.9.7
2、初始化配置文件
1mkdir -p ./rocketmq-namesrv-1/logs \
2./rocketmq-namesrv-1/store \
3./rocketmq-master-1/logs \
4./rocketmq-master-1/store
1vim ./rocketmq-namesrv-1/namesrv.properties
2
3# 编辑以下内容
4
5# NameServer 存储路径
6storePathRootDir=/home/recharge/local/rocketmq-namesrv-1/store/
7# NameServer 监听端口(默认为 9876)
8listenPort=9876
9# Broker 心跳超时时间(毫秒)
10waitTimeMillsInHeartbeat=30000
1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-1/broker.conf
2vim ./rocketmq-master-1/broker.conf
3
4# 编辑以下内容
5brokerClusterName = DefaultCluster
6brokerName = broker-a
7brokerId = 0
8#表示几点做消息删除动作,默认是凌晨4点
9deleteWhen = 04
10#在磁盘上保留消息的时长,单位是小时
11fileReservedTime = 48
12#有三个值:SYNC_MASTER,ASYNC_MASTER,SLAVE;
13brokerRole = ASYNC_MASTER
14#ASYNC_FLUSH异步刷盘,SYNC_FLUSH表示同步刷盘;SYNC_FLUSH消息写入磁盘后才返回成功状态,ASYNC_FLUSH不需要;
15flushDiskType = ASYNC_FLUSH
16
17#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
18brokerIP1=192.168.110.10
19
20namesrvAddr=127.0.0.1:9876
21
22listenPort=10911
23storePathRootDir=/home/recharge/local/rocketmq-master-1/store
24storePathCommitLog=/home/recharge/local/rocketmq-master-1/store/commitlog
25storePathConsumerQueue=/home/recharge/local/rocketmq-master-1/store/consumequeue
26storePathIndex=/home/recharge/local/rocketmq-master-1/store/index
27
28autoCreateTopicEnable=true
3、运行服务
1#运行nameserver
2nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-1/namesrv.properties &> ./rocketmq-namesrv-1/logs/namesrv.log &
3tail -fn 100 ./rocketmq-namesrv-1/logs/namesrv.log
4#输出:The Name Server boot success. serializeType=JSON
5
6#运行broker 之前如果在测试环境,内存不足时,需要更新JVM 启动参数
7vim ./rocketmq-4.9.7/bin/runbroker.sh
8# JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g"
9# 改为:JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"
10
11# 运行broker
12nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-1/broker.conf &> ./rocketmq-master-1/logs/broker.log &
13tail -fn 100 ./rocketmq-master-1/logs/broker.log
14# 输出:The broker[broker-a, 192.168.0.120:10911] boot success. serializeType=JSON
15
16#检查进程
17ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
18ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
19
20#关闭服务
21sh ./rocketmq-4.9.7/bin/mqshutdown broker
22sh ./rocketmq-4.9.7/bin/mqshutdown namesrv
跟着之前的我分享的文章教程 Java 客户端连接 RocketMQ ,就可以开始本地使用啦。
集群部署
单机部署一般用于本地或测试,生产环境不推荐使用,原因有两点:
1、broker服务器机械故障会导致该节点消息永久丢失,无法恢复,采用1主1从节同步数据;
2、broker单节点故障会导致服务不可用,采用多个主节点部署方式;
3、nameserver 单节点故障导致网络连接异常,采用多个节点部署(注册中心是无状态的)。
多节点存储方案
nameserver 角色 |
监听端口 |
存储路径 |
---|---|---|
server-1 |
9876 |
/home/recharge/local/rocketmq-namesrv-1/store/ |
server-1 |
9877 |
/home/recharge/local/rocketmq-namesrv-2/store/ |
server-3 |
9878 |
/home/recharge/local/rocketmq-namesrv-3/store/ |
Broker 节点
Broker 角色 | brokerName | brokerId |
监听端口 |
存储路径 |
---|---|---|---|---|
Master-1 |
broker-a |
0 |
10911 |
/home/recharge/local/rocketmq-master-1/store |
Slave-1 |
broker-a |
1 |
11911 |
/home/recharge/local/rocketmq-slave-1/store |
Master-2 |
broker-b |
0 |
10912 |
/home/recharge/local/rocketmq-master-2/store |
Slave-2 |
broker-b |
1 |
11912 |
/home/recharge/local/rocketmq-slave-1/store |
首先参考单机部署方案,将二进制下载到本地。部署方案将分为两步:安装 nameserver 集群 和 安装 Master-Slave 集群
安装 nameserver 集群
1、初始化文件目录
1cd /home/recharge/local
2
3mkdir -p ./rocketmq-namesrv-1/logs \
4./rocketmq-namesrv-1/store \
5./rocketmq-namesrv-2/logs \
6./rocketmq-namesrv-2/store \
7./rocketmq-namesrv-3/logs \
8./rocketmq-namesrv-3/store
2、初始化 namesrver 配置文件
1vim ./rocketmq-namesrv-1/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-1/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9876
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000
1vim ./rocketmq-namesrv-2/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-2/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9877
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000
1vim ./rocketmq-namesrv-3/namesrv.properties
2
3# NameServer 存储路径
4storePathRootDir=/home/recharge/local/rocketmq-namesrv-3/store/
5# NameServer 监听端口(默认为 9876)
6listenPort=9878
7# Broker 心跳超时时间(毫秒)
8waitTimeMillsInHeartbeat=30000
3、运行 nameserver 集群
1#运行nameserver1
2nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-1/namesrv.properties &> ./rocketmq-namesrv-1/logs/namesrv.log &
3tail -fn 100 ./rocketmq-namesrv-1/logs/namesrv.log
4#输出:The Name Server boot success. serializeType=JSON
5ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
6
7#运行nameserver2
8nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-2/namesrv.properties &> ./rocketmq-namesrv-2/logs/namesrv.log &
9tail -fn 100 ./rocketmq-namesrv-2/logs/namesrv.log
10#输出:The Name Server boot success. serializeType=JSON
11ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
12
13#运行nameserver3
14nohup sh ./rocketmq-4.9.7/bin/mqnamesrv -c ./rocketmq-namesrv-3/namesrv.properties &> ./rocketmq-namesrv-3/logs/namesrv.log &
15tail -fn 100 ./rocketmq-namesrv-3/logs/namesrv.log
16#输出:The Name Server boot success. serializeType=JSON
17ps -aux|grep org.apache.rocketmq.namesrv.NamesrvStartup
4、关闭服务
1sh ./rocketmq-4.9.7/bin/mqshutdown namesrv
安装Master-Slave集群
1、初始化文件目录
1cd /home/recharge/local
2
3mkdir -p ./rocketmq-master-1/logs \
4./rocketmq-master-1/store \
5./rocketmq-slave-1/logs \
6./rocketmq-slave-1/store
7
8mkdir -p ./rocketmq-master-2/logs \
9./rocketmq-master-2/store \
10./rocketmq-slave-2/logs \
11./rocketmq-slave-2/store
2、初始化master-slave配置文件
1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-1/broker.conf
2vim ./rocketmq-master-1/broker.conf
3
4# 编辑以下内容
5brokerClusterName=rocketmq-cluster
6brokerName=broker-a
7brokerId=0
8deleteWhen=04
9fileReservedTime=48
10brokerRole=ASYNC_MASTER
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16#Broker 外部服务端口(默认 10911) → 用于处理消息存储和传输
17listenPort=10911
18#Broker 高速存储同步端口(默认 10912) → 用于 Master-Slave 复制
19haListenPort=10912
20#Broker Web 控制台端口(默认 10909)
21dLegerPort=10909
22
23storePathRootDir=/home/recharge/local/rocketmq-master-1/store
24storePathCommitLog=/home/recharge/local/rocketmq-master-1/store/commitlog
25storePathConsumerQueue=/home/recharge/local/rocketmq-master-1/store/consumequeue
26storePathIndex=/home/recharge/local/rocketmq-master-1/store/index
27storePathConsumerOffset=/home/recharge/local/rocketmq-master-1/store/consumerOffset
28storePathConsumerOffset=/home/recharge/local/rocketmq-master-1/store/consumerOffset
29
30autoCreateTopicEnable=true
1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-slave-1/broker.conf
2vim ./rocketmq-slave-1/broker.conf
3
4# 编辑以下内容
5brokerClusterName=rocketmq-cluster
6brokerName=broker-a
7brokerId=1
8deleteWhen=04
9fileReservedTime=48
10brokerRole=SLAVE
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16listenPort=20911
17haListenPort=20912
18dLegerPort=20909
19
20storePathRootDir=/home/recharge/local/rocketmq-slave-1/store
21storePathCommitLog=/home/recharge/local/rocketmq-slave-1/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-slave-1/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-slave-1/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-slave-1/store/consumerOffset
25
26autoCreateTopicEnable=false
1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-master-2/broker.conf
2vim ./rocketmq-master-2/broker.conf
3
4# 编辑以下内容
5brokerClusterName=rocketmq-cluster
6brokerName=broker-b
7brokerId=0
8deleteWhen=04
9fileReservedTime=48
10brokerRole=ASYNC_MASTER
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
14brokerIP1=192.168.110.10
15
16listenPort=11911
17haListenPort=11912
18dLegerPort=11909
19
20storePathRootDir=/home/recharge/local/rocketmq-master-2/store
21storePathCommitLog=/home/recharge/local/rocketmq-master-2/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-master-2/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-master-2/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-master-2/store/consumerOffset
25
26autoCreateTopicEnable=true
1cp ./rocketmq-4.9.7/conf/broker.conf ./rocketmq-slave-2/broker.conf
2vim ./rocketmq-slave-2/broker.conf
3
4# 编辑以下内容
5brokerClusterName=rocketmq-cluster
6brokerName=broker-b
7brokerId=1
8deleteWhen=04
9fileReservedTime=48
10brokerRole=SLAVE
11namesrvAddr=127.0.0.1:9876;127.0.0.1:9877;127.0.0.1:9878
12
13listenPort=21911
14haListenPort=21912
15dLegerPort=21909
16#本机IP NameServer 和消费者都会使用这个 IP 访问 Broker
17brokerIP1=192.168.110.10
18
19
20storePathRootDir=/home/recharge/local/rocketmq-slave-2/store
21storePathCommitLog=/home/recharge/local/rocketmq-slave-2/store/commitlog
22storePathConsumerQueue=/home/recharge/local/rocketmq-slave-2/store/consumequeue
23storePathIndex=/home/recharge/local/rocketmq-slave-2/store/index
24storePathConsumerOffset=/home/recharge/local/rocketmq-slave-2/store/consumerOffset
25
26autoCreateTopicEnable=false
3、运行服务
1# 运行broker-a-0
2nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-1/broker.conf &> ./rocketmq-master-1/logs/broker.log &
3tail -fn 100 ./rocketmq-master-1/logs/broker.log
4ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
5
6# 运行broker-a-1
7nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-slave-1/broker.conf &> ./rocketmq-slave-1/logs/broker.log &
8tail -fn 100 ./rocketmq-slave-1/logs/broker.log
9ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
10
11
12# 运行broker-b-0
13nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-master-2/broker.conf &> ./rocketmq-master-2/logs/broker.log &
14tail -fn 100 ./rocketmq-master-2/logs/broker.log
15ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
16
17
18# 运行broker-b-1
19nohup sh ./rocketmq-4.9.7/bin/mqbroker -c ./rocketmq-slave-2/broker.conf &> ./rocketmq-slave-2/logs/broker.log &
20tail -fn 100 ./rocketmq-slave-2/logs/broker.log
21ps -aux|grep org.apache.rocketmq.broker.BrokerStartup
4、关闭服务
1sh ./rocketmq-4.9.7/bin/mqshutdown broker
至此,RocketMQ 集群环境部署完成,生产环境就可以投入使用~