Redis cluster（八）-part 1

最新推荐文章于 2024-06-27 02:15:10 发布

原创最新推荐文章于 2024-06-27 02:15:10 发布 · 457 阅读

0 ·

CC 4.0 BY-SA版权

java 同时被 3 个专栏收录

46 篇文章

订阅专栏

redis

15 篇文章

订阅专栏

Docker

5 篇文章

订阅专栏

本文详细介绍了 Redis Cluster 的架构特点，包括节点自动发现、故障转移机制等，并提供了使用 Docker 和 Ruby 安装部署 Redis Cluster 的步骤。此外，还分享了通过 Java 进行集群测试的具体代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

一:关于redis cluster

redis cluster的现状

目前redis支持的cluster特性

1):节点自动发现

2):slave->master 选举,集群容错

3):Hot resharding:在线分片

4):进群管理:cluster xxx

5):基于配置(nodes-port.conf)的集群管理

6):ASK 转向/MOVED 转向机制.

redis cluster 架构

1)redis-cluster架构图

架构细节:

(1)所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.

(2)节点的fail是通过集群中超过半数的节点检测失效时才生效.

(3)客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可

(4)redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value

2) redis-cluster选举:容错

(1)领着选举过程是集群中所有master参与,如果半数以上master节点与master节点通信超过(cluster-node-timeout),认为当前master节点挂掉.

(2):什么时候整个集群不可用(cluster_state:fail),当集群不可用时,所有对集群的操作做都不可用，收到((error) CLUSTERDOWN The cluster is down)错误

a:如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成进群的slot映射[0-16383]不完成时进入fail状态.

b:如果进群超过半数以上master挂掉，无论是否有slave集群进入fail状态.

二:redis cluster的使用

1:部署和安装redis cluster

～～这里用的是 docker：

一、环境

os：centos7
redis：3.2.9
gem-redis：3.2.2（可去下载最新版噢）

二、搭建集群

1、本机下载redis-3.2.9.tar.gz

redis官网：https://redis.io/download

2、解压安装redis（最少准备 6 个~3m3s）

这个可以参考这个文章地址

3、安装ruby依赖

yum install ruby rubygems -y

4、本机下载安装gem-redis

下载：https://rubygems.org/gems/redis/versions/3.2.2
安装：gem install -l /opt/redis-3.2.2.gem

5、配置redis－config

1)redis配置文件结构:

使用包含(include)把通用配置和特殊配置分离,方便维护

.2)redis通用配置.

#GENERAL  
daemonize no  
tcp-backlog 511  
timeout 0  
tcp-keepalive 0  
loglevel notice  
databases 16  
slave-serve-stale-data yes  
#slave只读  
slave-read-only yes  
#not use default  
repl-disable-tcp-nodelay yes  
slave-priority 100  
#打开aof持久化  
appendonly no  
#每秒一次aof写  
appendfsync everysec  
#关闭在aof rewrite的时候对新的写操作进行fsync  
no-appendfsync-on-rewrite yes  
auto-aof-rewrite-min-size 64mb  
lua-time-limit 5000  
#打开redis集群  
cluster-enabled yes  
#节点互连超时的阀值  
cluster-node-timeout 15000  
cluster-migration-barrier 1  
slowlog-log-slower-than 10000  
slowlog-max-len 128  
notify-keyspace-events ""  
hash-max-ziplist-entries 512  
hash-max-ziplist-value 64  
list-max-ziplist-entries 512  
list-max-ziplist-value 64  
set-max-intset-entries 512  
zset-max-ziplist-entries 128  
zset-max-ziplist-value 64  
activerehashing yes  
client-output-buffer-limit normal 0 0 0  
client-output-buffer-limit slave 256mb 64mb 60  
client-output-buffer-limit pubsub 32mb 8mb 60  
hz 10  
aof-rewrite-incremental-fsync yes  
protected-mode no

3)redis特殊配置.（例如6379的配置）

/sys/fs/cgroup/ 为docker 挂载地址

#包含通用配置  
include /sys/fs/cgroup/newconfig/redis_cluster_common.conf
#监听tcp端口  
port 6379
#rdb文件,只用于动态添加slave过程  
dbfilename "dump6379.rdb"
pidfile "/sys/fs/cgroup/redis_6379.pid"
logfile "/sys/fs/cgroup/redis_6379.log"
dbfilename "dump6379.rdb"
dir "/sys/fs/cgroup"
#aof存储文件  
appendfilename "appendonly-6379.aof"  

#最大可用内存  
maxmemory 100m  
#内存耗尽时采用的淘汰策略:  
# volatile-lru -> remove the key with an expire set using an LRU algorithm  
# allkeys-lru -> remove any key accordingly to the LRU algorithm  
# volatile-random -> remove a random key with an expire set  
# allkeys-random -> remove a random key, any key  
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)  
# noeviction -> don't expire at all, just return an error on write operations  
maxmemory-policy allkeys-lru  

#cluster配置文件(启动自动生成)  
cluster-config-file nodes-6379.conf  
#部署在同一机器的redis实例，把auto-aof-rewrite搓开，
#防止瞬间fork所有redis进程做rewrite,占用大量内存 80-100
auto-aof-rewrite-percentage 80

6、启动6个redis实例

怎么启动就不说了

7、创建集群

./redis-trib.rb  create --replicas 1 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:6382 127.0.0.1:6383 127.0.0.1:6384

记得输入：yes

[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:6382 127.0.0.1:6383 127.0.0.1:6384  
>>> Creating cluster  
>>> Performing hash slots allocation on 6 nodes...  
Using 3 masters:  
127.0.0.1:6379  
127.0.0.1:6380  
127.0.0.1:6381  
Adding replica 127.0.0.1:6382 to 127.0.0.1:6379  
Adding replica 127.0.0.1:6383 to 127.0.0.1:6380  
Adding replica 127.0.0.1:6384 to 127.0.0.1:6381  
M: dfd510594da614469a93a0a70767ec9145aefb1a 127.0.0.1:6379  
   slots:0-5460 (5461 slots) master  
M: e02eac35110bbf44c61ff90175e04d55cca097ff 127.0.0.1:6380  
   slots:5461-10922 (5462 slots) master  
M: 4385809e6f4952ecb122dbfedbee29109d6bb234 127.0.0.1:6381  
   slots:10923-16383 (5461 slots) master  
S: ec02c9ef3acee069e8849f143a492db18d4bb06c 127.0.0.1:6382  
   replicates dfd510594da614469a93a0a70767ec9145aefb1a  
S: 83e5a8bb94fb5aaa892cd2f6216604e03e4a6c75 127.0.0.1:6383  
   replicates e02eac35110bbf44c61ff90175e04d55cca097ff  
S: 10c097c429ca24f8720986c6b66f0688bfb901ee 127.0.0.1:6384  
   replicates 4385809e6f4952ecb122dbfedbee29109d6bb234  
Can I set the above configuration? (type 'yes' to accept): yes  
>>> Nodes configuration updated  
>>> Assign a different config epoch to each node  
>>> Sending CLUSTER MEET messages to join the cluster  
Waiting for the cluster to join......  
>>> Performing Cluster Check (using node 127.0.0.1:6379)  
M: dfd510594da614469a93a0a70767ec9145aefb1a 127.0.0.1:6379  
   slots:0-5460 (5461 slots) master  
M: e02eac35110bbf44c61ff90175e04d55cca097ff 127.0.0.1:6380  
   slots:5461-10922 (5462 slots) master  
M: 4385809e6f4952ecb122dbfedbee29109d6bb234 127.0.0.1:6381  
   slots:10923-16383 (5461 slots) master  
M: ec02c9ef3acee069e8849f143a492db18d4bb06c 127.0.0.1:6382  
   slots: (0 slots) master  
   replicates dfd510594da614469a93a0a70767ec9145aefb1a  
M: 83e5a8bb94fb5aaa892cd2f6216604e03e4a6c75 127.0.0.1:6383  
   slots: (0 slots) master  
   replicates e02eac35110bbf44c61ff90175e04d55cca097ff  
M: 10c097c429ca24f8720986c6b66f0688bfb901ee 127.0.0.1:6384  
   slots: (0 slots) master  
   replicates 4385809e6f4952ecb122dbfedbee29109d6bb234  
[OK] All nodes agree about slots configuration.  
>>> Check for open slots...  
>>> Check slots coverage...  
[OK] All 16384 slots covered.

8、java简单测试代码

import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;

public class RedisClusterTest {
	 public static void main(String[] args) throws IOException {
		 JedisCluster jedisCluster=null;
		 try{
			 JedisPoolConfig config = new JedisPoolConfig();  
			    config.setMaxTotal(1000);
			    config.setMaxIdle(100);
		       config.setTestOnBorrow(true);
			      
			    Set<HostAndPort> nodes = new HashSet<HostAndPort>();  
			    String host="192.168.1.107";
			    HostAndPort hostAndPort = new HostAndPort(host,6379);  
			    HostAndPort hostAndPort1 = new HostAndPort(host,6380);  
			    HostAndPort hostAndPort2 = new HostAndPort(host,6381);  
			    HostAndPort hostAndPort3 = new HostAndPort(host,6382);  
			    HostAndPort hostAndPort4 = new HostAndPort(host,6383);  
			    HostAndPort hostAndPort5 = new HostAndPort(host,6384);  
			     		    		    
			    nodes.add(hostAndPort);  
			    nodes.add(hostAndPort1);  
			    nodes.add(hostAndPort2);  
			    nodes.add(hostAndPort3);  
			    nodes.add(hostAndPort4);  
			    nodes.add(hostAndPort5);  
			 
			 
			     jedisCluster=new JedisCluster(nodes,5000,config);
			   jedisCluster.set("kkk", "1");
			   String kkk = jedisCluster.get("kkk");  
			   System.out.println(kkk);
		

		 }finally {
			if(null!=jedisCluster)
				   jedisCluster.close();
		}
		  
	}
 
	  
}