当前位置: 首页>>系统&架构>>正文

LINUX下Redis集群的安装和配置(原创)

@ ▣系统&架构 ♢, , , ,

在大数据的浪潮下，redis只提供主从分布模式越来越显得捉襟见肘了。2015年3月底，Redis-Cluster终于出了正式版本，真可谓千呼万唤始出来啊。下面我们就来看看如何一步一步安装和配置Redis集群。
全文共分9个安装步骤进行介绍，提供了所有安装相关的自动化脚本源码，这些脚本可以通过配置文件扩展到多台机器多个实例的redis-cluster安装。 操作系统：Centos6.4 Redis版本: 3.0 机器(64位)：3台机器用于安装redis集群+1台机器做总控 网路环境: 4台机器在同一个局域网 集群拓扑结构：9个实例，3主6从，每个主2个从节点。 1. 首先到redis官网下载redis3.0版本, 程序文件包居然只有1.3MB，赞一个。

wget http://download.redis.io/releases/redis-3.0.0.tar.gz

2. 准备可执行程序：在总控机器上解压&编译

tar xzvf redis-3.0.0.tar.gz
cd redis-3.0.0
make

这个时候在src目录下已经生成了redis的可执行程序了(绿色部分)

 

3.  准备redis的配置文件: 这里根据:http://redis.io/topics/cluster-tutorial写了个最简单版本的(文件名redis.conf)：

port 30000
daemonize yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly no


loglevel warning
logfile ../logs/redis.log
pidfile ../logs/redis.pid

timeout 60
tcp-keepalive 60

rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir ../logs/

save 900 1
save 300 10
save 60 100000

maxclients 65536
slave-read-only yes

4. 打包程序和配置文件：使用如下的脚本，打包之后方便拷贝到三台待安装机器上部署(文件名tar_redis.sh).

#!/bin/bash

#打包redis程序和配置

if [ $# -ne 0 ]
then
	echo "Usage: $0"
	exit 1
fi

rm -rf redis-server.tar.gz
mkdir -p redis-server
mkdir -p redis-server/conf
mkdir -p redis-server/sbin
mkdir -p redis-server/logs
cp -f ./redis.conf  redis-server/conf/
cd ./redis-3.0.0/src/
cp -f mkreleasehdr.sh redis-benchmark redis-check-aof redis-check-dump redis-cli \
	redis-sentinel redis-server redis-trib.rb ../../redis-server/sbin/
cd -
tar czvf redis-server.tar.gz redis-server

exit 0

sh tar_redis.sh执行后生成程序和配置文件包redis-server.tar.gz

5. 分发程序和配置文件包到3台待安装安装的机器, 这里我们也写了个自动脚本dist_redis.sh. (使用这个脚本需要先安装sshpass做远程自动登录[见另一篇博文Linux自动密码登陆利器sshpass中的安装和问题说明], sshpass是建立信任关系之外的另一种选择).该脚本的功能是：

  • 删除就的备份目录并备份当前目录
  • 拷贝程序包到制定目录 并 解压
  • 设置配置文件中的端口
  • 执行方法: sh dist_redis.sh ipportlist.txt 
#!/bin/bash
if [ $# -ne 1 ]
then
	echo "Usage: $0 iplistFile"
	exit 1
fi

#自动分发redis程序和配置到所有机器[见输入的iplistFile]
username='root'  
password='123456'  
dist_dir=/home/redis/redis-server/
exe_files="./redis-server.tar.gz"
echo ${dist_dir}
echo ${exe_files}

while read ipport
do
	ip=`echo ${ipport} |cut -d: -f1`
	port=`echo ${ipport} | cut -d: -f2`
	curr_dir="`echo ${dist_dir}|sed 's:/$::g'`-${port}"
	bak_dir=`echo "${curr_dir}" | sed 's:/$::g'`_bak
	echo "process host:${ip}:${port} at ${curr_dir}============================================="
	echo "step 1: rm old bak:${bak_dir}------------------------------------------"
	sshpass -p ${password} ssh ${username}@${ip} "rm -rf  ${bak_dir} 0</dev/null" 0</dev/null
	echo $?

	echo "step 2: backup data-------------------------------------"
	sshpass -p ${password} ssh ${username}@${ip} "mv -f ${curr_dir} ${bak_dir} 2>/dev/null 1>&2 0</dev/null" 0</dev/null
	echo $?

	echo "step 3: mkdir dir:${curr_dir}---------------------------"
	sshpass -p ${password} ssh ${username}@${ip} "mkdir -p ${curr_dir}  0</dev/null" 0</dev/null
	echo $?
	
	echo "step 4: copy tarfiles------------------------------------"
	sshpass -p ${password} scp ${exe_files} ${username}@${ip}:${curr_dir}
	echo $?

	echo "step 5: decompress files---------------------------------"
	sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir} && tar xzvf ${exe_files} 1>/dev/null && mv -f redis-server/* . && rm -rf redis-server 0</dev/null" 0</dev/null
	echo $?

	echo "step 6.${i}: update conf port ---------------------------------"
	sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir} && sed -i 's/^port[ 0-9]\+$/port ${port}/g' ${curr_dir}/conf/redis.conf 0</dev/null" 0</dev/null
	echo $?
done < $1


exit 0

这个脚本的输入 ipportlist.txt文件格式为：

10.71.48.113:12000
10.71.48.114:12001
10.71.48.115:12002
10.71.48.114:12003
10.71.48.113:12004
10.71.48.115:12005
10.71.48.115:12006
10.71.48.113:12007
10.71.48.114:12008

 
6. 在启动集群之前，我们需要关闭待安装机器的防火墙。由于redis集群每个实例会开一个服务侦听端口和一个集群内通信端口，为了方便我们这里简单处理，直接写了个脚本把防火墙关闭了。也可以去iptables上配置好开放相应的端口。开关防火墙的脚本:

  • 执行方法: cat iplist.txt | sh firewall_control.sh start 
#!/bin/bash
if [ $# -ne 1 ]
then
	echo "Usage: $0 start|stop < iplistFile"
	exit 1
fi

username='root'
password='123456'
while read ip
do
	echo "at ${ip}"
	if [ "${1}X" == "startX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "service iptables start 0</dev/null" 0</dev/null
	elif [ "${1}X" == "stopX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "service iptables stop 0</dev/null" 0</dev/null
	else
		echo "invalid command!"
	fi
done 
echo "done."
exit 0

iplist.txt文件格式为：

10.71.48.113
10.71.48.114
10.71.48.115

7. 开启redis程序: 遍历所有实例的目录，cd到sbin下使用./redis-server -c ../conf/redis.conf启动redis实例, 批处理脚本redis_control.sh如下，包括了redis的开启/关闭/重启/存活检测。

  • 启动： sh redis_control.sh ipportlist.txt start
  • 关闭:     sh redis_control.sh ipportlist.txt stop 
#!/bin/bash
if [ $# -ne 2 ]
then
	echo "Usage: $0 iplistFile start|stop|restart|check"
	exit 1
fi

#启动所有redis实例
username='root'
password='123456'
dist_dir=/home/redis/redis-server
exe_dir="./redis-server"
echo ${dist_dir}

command=$2

while read ipport
do
	ip=`echo ${ipport} |cut -d: -f1`
	port=`echo ${ipport} | cut -d: -f2`
	curr_dir="`echo ${dist_dir}|sed 's:/$::g'`-${port}"
	bak_dir=`echo "${curr_dir}" | sed 's:/$::g'`_bak
	echo "process host:${ip}:${port} at ${curr_dir}:${command}============================================="
	if [ "${command}X" == "startX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir}/sbin && ./redis-server ../conf/redis.conf" 0</dev/null
	elif [ "${command}X" == "stopX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir} && cat ./logs/redis.pid |xargs kill -9" 0</dev/null	
	elif [ "${command}X" == "restartX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir} && cat ./logs/redis.pid |xargs kill -9" 0</dev/null
		sshpass -p ${password} ssh ${username}@${ip} "cd ${curr_dir}/sbin && ./redis-server ../conf/redis.conf" 0</dev/null
	elif [ "${command}X" == "checkX" ]
	then
		sshpass -p ${password} ssh ${username}@${ip} "ps -ef |grep redis|grep ${port}|grep -v grep" 0</dev/null
	else
		echo "invalid command!"
	fi
	echo $?
done < $1


exit 0

8. 执行上面的start命令之后，只是启动了所有的实例，这些实例还没有互相连接起来组成集群。下面我们来看看如果把这些机器连接成集群。实际上redis官方提供了ruby脚本: ./redis-3.0.0/src/redis-trib.rb,来方便我们创建redis集群，不过需要安装相关的工具:

  • 安装ruby: yum install -y ruby
  • 安装rubygems: yum install -y rubygems , 不然会报错: `require’: no such file to load — rubygems (LoadError)
  • gem默认的源比较慢，我们删除默认的源并添加一个源：

    gem sources -r http://rubygems.org/
gem sources -a http://ruby.taobao.org/
  • 安装ruby-redis:  gem  install redis, 不然会报错：in `gem_original_require’: no such file to load — redis (LoadError)

9.  安装好上述相关工具后就可以直接使用脚本redis-trib.rb安装了。同样，这里我们也对redis-trib.rb做了简单的封装，方便对参数的控制。封装之后的安装脚本为: redis_cluster_create.sh

  • 执行: sh redis_cluster_create.sh ipportlist.txt
  • 安装中根据提示输入yes即可
  • 注意: –replicas 2表示每个主节点两个从节点 
#!/bin/bash
if [ $# -ne 1 ]
then
	echo "Usage: $0 ipportlist.txt"
	exit 1
fi

#创建集群
all_addr=""
rb="./redis-3.0.0/src/redis-trib.rb"
all_addr=`xargs -n9 < $1`
echo ${all_addr}
echo "create cluster..."
${rb} create --replicas 2 ${all_addr}
echo "done."
exit 0

执行结果：

[root@xx106 redis-install]# sh redis_cluster_create.sh ipportlist.txt 
10.71.48.113:12000 10.71.48.114:12001 10.71.48.115:12002 10.71.48.114:12003 10.71.48.113:12004 10.71.48.115:12005 10.71.48.115:12006 10.71.48.113:12007 10.71.48.114:12008
create cluster...
>>> Creating cluster
Connecting to node 10.71.48.113:12000: OK
Connecting to node 10.71.48.114:12001: OK
Connecting to node 10.71.48.115:12002: OK
Connecting to node 10.71.48.114:12003: OK
Connecting to node 10.71.48.113:12004: OK
Connecting to node 10.71.48.115:12005: OK
Connecting to node 10.71.48.115:12006: OK
Connecting to node 10.71.48.113:12007: OK
Connecting to node 10.71.48.114:12008: OK
>>> Performing hash slots allocation on 9 nodes...
Using 3 masters:
10.71.48.115:12002
10.71.48.114:12001
10.71.48.113:12000
Adding replica 10.71.48.114:12003 to 10.71.48.115:12002
Adding replica 10.71.48.113:12004 to 10.71.48.115:12002
Adding replica 10.71.48.115:12005 to 10.71.48.114:12001
Adding replica 10.71.48.115:12006 to 10.71.48.114:12001
Adding replica 10.71.48.114:12008 to 10.71.48.113:12000
Adding replica 10.71.48.113:12007 to 10.71.48.113:12000
M: 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7 10.71.48.113:12000
   slots:10923-16383 (5461 slots) master
M: 29569c90e2d11722f676bc8c650167ec6e6fcbe1 10.71.48.114:12001
   slots:5461-10922 (5462 slots) master
M: bb9b7580b7b37b11ed1218af6e3c13794756a3f3 10.71.48.115:12002
   slots:0-5460 (5461 slots) master
S: ae8baf40cf3741f9d4b6f3e880ba984867f1e66b 10.71.48.114:12003
   replicates bb9b7580b7b37b11ed1218af6e3c13794756a3f3
S: 32962c506aa2e1a61c15b644a1750f0685739d7b 10.71.48.113:12004
   replicates bb9b7580b7b37b11ed1218af6e3c13794756a3f3
S: 7ce9a69167e3c882db81539c6f01eb3de43b1526 10.71.48.115:12005
   replicates 29569c90e2d11722f676bc8c650167ec6e6fcbe1
S: 3c949cb159bc2e74f49ccf1398ea6e3279d48656 10.71.48.115:12006
   replicates 29569c90e2d11722f676bc8c650167ec6e6fcbe1
S: f6e15fad28fe15e4546cb984f226c7540388e698 10.71.48.113:12007
   replicates 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7
S: 6b3dc056e05e06ad5fb186881a2ad201a5ba8bb0 10.71.48.114:12008
   replicates 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 10.71.48.113:12000)
M: 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7 10.71.48.113:12000
   slots:10923-16383 (5461 slots) master
M: 29569c90e2d11722f676bc8c650167ec6e6fcbe1 10.71.48.114:12001
   slots:5461-10922 (5462 slots) master
M: bb9b7580b7b37b11ed1218af6e3c13794756a3f3 10.71.48.115:12002
   slots:0-5460 (5461 slots) master
M: ae8baf40cf3741f9d4b6f3e880ba984867f1e66b 10.71.48.114:12003
   slots: (0 slots) master
   replicates bb9b7580b7b37b11ed1218af6e3c13794756a3f3
M: 32962c506aa2e1a61c15b644a1750f0685739d7b 10.71.48.113:12004
   slots: (0 slots) master
   replicates bb9b7580b7b37b11ed1218af6e3c13794756a3f3
M: 7ce9a69167e3c882db81539c6f01eb3de43b1526 10.71.48.115:12005
   slots: (0 slots) master
   replicates 29569c90e2d11722f676bc8c650167ec6e6fcbe1
M: 3c949cb159bc2e74f49ccf1398ea6e3279d48656 10.71.48.115:12006
   slots: (0 slots) master
   replicates 29569c90e2d11722f676bc8c650167ec6e6fcbe1
M: f6e15fad28fe15e4546cb984f226c7540388e698 10.71.48.113:12007
   slots: (0 slots) master
   replicates 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7
M: 6b3dc056e05e06ad5fb186881a2ad201a5ba8bb0 10.71.48.114:12008
   slots: (0 slots) master
   replicates 3bf182ba239438d96ef6ea4f3fa1a9b333d273d7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
done.

10. 运行：

[root@xx113 redis-server-12004]# ./sbin/redis-cli -c -p 12004
127.0.0.1:12004> set fuqingchuan abc
-> Redirected to slot [11708] located at 10.71.48.113:12000
OK
10.71.48.113:12000> set hello world
-> Redirected to slot [866] located at 10.71.48.115:12002
OK
10.71.48.115:12002>
本文由《纯净天空》出品。文章地址: https://vimsky.com/article/248.html，未经允许，请勿转载。