搭建高可用及负载均衡的Redis

项目决定数据存储使用Redis为主,Mysql只保存日志信息。需要对Redis的高可用及负载均衡做一个部署测试。测试使用虚拟机用。

测试环境

  • 机器

    • 母机:Ubuntu 4核 16G

    • 虚拟机:centos6.5-64 单核 1G 独立ip 3个

      虚拟机使用virtual box,centos为64位6.5.具体信息如下

      192.168.0.231   Redis-master
      
      192.168.0.231   Redis-slave
      
      192.168.0.239   Virtual IP
      
  • 软件

    • Reidis
    • Keepalived
  • 安装前配置

    虚拟机最小化安装,更新系统及安装必须组件

    $ yum install -y openssl openssl-devel kernel kernel-devel gcc wget tcl libnl-devel
    
    $ yum upgrade
    
    $ reboot  //重启系统
    

安装Keepalived

  • 安装keepalived 下载地址:http://www.keepalived.org/software/keepalived-1.2.13.tar.gz 安装:

    $ wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz
    
    $ tar -zxvf keepalived-1.2.13.tar.gz
    
    $ cd keepalived-1.2.13
    
    $ ln -s /usr/src/kernels/`uname -r`-`uname -m`/ /usr/src/linux
    
    $ ./configure  --prefix=/usr/local/keepalived  --sysconf=/etc
    
    $ chkconfig --add keepalived
    
    //查看是否成功
    
    $ service keepalived status
    
  • 安装redis

    下载Redis,目前Redis的稳定版本是2.8.17,下载地址Redis-2.8.17

    $ wget http://download.redis.io/releases/redis-2.8.17.tar.gz
    
    $ tar -zxvf redis-2.8.17.tar.gz
    
    $ cd redis-2.8.17
    
    $ make && make test
    
    $ mkdir -p /usr/local/redis/{bin,var,etc}
    
    $ cp ./src/{redis-benchmark,redis-cli,redis-server,redis-check-dump,redis-check-aof} /usr/local/redis/bin/
    
    $ cp /usr/local/redis/bin/* /usr/local/bin/
    

配置

  • Master配置

    先配置好Redis,主从复制成功之后再行配置Keepalived。

    • Master-Redis配置

      $ vim /usr/local/redis/etc/redis.conf
      

      修改如下值

      daemonize yes
      pidfile /var/run/redis.pid
      port 6379
      tcp-backlog 511
      timeout 0
      tcp-keepalive 0
      loglevel notice
      logfile "/usr/local/redis/log/redis.log"
      databases 16
      save 900 1
      save 300 10
      save 60 10000
      stop-writes-on-bgsave-error yes
      rdbcompression yes
      rdbchecksum yes
      dbfilename dump.rdb
      dir /usr/local/redis/data
      slave-serve-stale-data yes
      slave-read-only no
      repl-disable-tcp-nodelay no
      slave-priority 100
      appendonly yes
      appendfilename "appendonly.aof"
      appendfsync everysec
      no-appendfsync-on-rewrite no
      auto-aof-rewrite-percentage 100
      auto-aof-rewrite-min-size 64mb
      lua-time-limit 5000
      slowlog-log-slower-than 10000
      slowlog-max-len 128
      notify-keyspace-events ""
      hash-max-ziplist-entries 512
      hash-max-ziplist-value 64
      list-max-ziplist-entries 512
      list-max-ziplist-value 64
      set-max-intset-entries 512
      zset-max-ziplist-entries 128
      zset-max-ziplist-value 64
      activerehashing yes
      client-output-buffer-limit normal 0 0 0
      client-output-buffer-limit slave 256mb 64mb 60
      client-output-buffer-limit pubsub 32mb 8mb 60
      hz 10
      aof-rewrite-incremental-fsync yes
      
    • Master-Keepalived配置

      global_defs {
          router_id redis
      }
      vrrp_script chk_redis {
          script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本
          interval 2                  ###监控时间
          timeout  2                                  ###超时时间
          fall     3                  ###重试次数
      }
      vrrp_instance redis {
        state MASTER                  ###设置为MASTER
        interface eth0              ###监控网卡
        virtual_router_id 52
        priority 101              ###权重值
        authentication {
               auth_type PASS          ###加密
               auth_pass redis        ###密码
        }
        track_script {
          chk_redis              ###执行上面定义的chk_redis
        }
        virtual_ipaddress {
             192.168.0.239            ###VIP
        }
        notify_master /etc/keepalived/scripts/redis_master.sh
        notify_backup /etc/keepalived/scripts/redis_backup.sh
        notify_fault  /etc/keepalived/scripts/redis_fault.sh
        notify_stop   /etc/keepalived/scripts/redsi_stop.sh
      }
      

      也可在本地编辑好后使用scp上传到服务器。 Master-keepalived脚本redis_master.sh:

        #!/bin/bash
        REDISCLI="/usr/local/redis/bin/redis-cli"
        LOGFILE="/usr/local/redis/var/keepalived-redis-state.log"
        echo "[master]" >> $LOGFILE
        date >> $LOGFILE
        echo "Being master...." >> $LOGFILE 2>&1
        echo "Run SLAVEOF cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE  2>&1
        sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
        echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
      

      Master-keepalived脚本redis_backup.sh:

        #!/bin/bash
        REDISCLI="/usr/local/redis/bin/redis-cli"
        LOGFILE="/usr/local/redis/log/keepalived-redis-state.log"
        echo "[backup]" >> $LOGFILE
        date >> $LOGFILE
        echo "Being slave...." >> $LOGFILE 2>&1
        sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
        echo "Run SLAVEOF cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF 192.168.0.232 6379 >> $LOGFILE  2>&1
      
  • Backup配置

    同样先配置好Redis,主从复制成功之后再行配置Keepalived。

    • Backup-Redis配置

      $ vim /usr/local/redis/etc/redis.conf
      

      修改如下值

      daemonize yes
      pidfile /var/run/redis.pid
      port 6379
      tcp-backlog 511
      timeout 0
      tcp-keepalive 0
      loglevel notice
      logfile "/usr/local/redis/log/redis.log"
      databases 16
      save 900 1
      save 300 10
      save 60 10000
      stop-writes-on-bgsave-error yes
      rdbcompression yes
      rdbchecksum yes
      dbfilename dump.rdb
      dir /usr/local/redis/data
      slave-serve-stale-data yes
      slave-read-only no
      repl-disable-tcp-nodelay no
      slave-priority 100
      appendonly yes
      appendfilename "appendonly.aof"
      appendfsync everysec
      no-appendfsync-on-rewrite no
      auto-aof-rewrite-percentage 100
      auto-aof-rewrite-min-size 64mb
      lua-time-limit 5000
      slowlog-log-slower-than 10000
      slowlog-max-len 128
      notify-keyspace-events ""
      hash-max-ziplist-entries 512
      hash-max-ziplist-value 64
      list-max-ziplist-entries 512
      list-max-ziplist-value 64
      set-max-intset-entries 512
      zset-max-ziplist-entries 128
      zset-max-ziplist-value 64
      activerehashing yes
      client-output-buffer-limit normal 0 0 0
      client-output-buffer-limit slave 256mb 64mb 60
      client-output-buffer-limit pubsub 32mb 8mb 60
      hz 10
      aof-rewrite-incremental-fsync yes
      SLAVEOF 192.168.0.231 6379
      
    • Backup-Keepalived配置

      global_defs {
          router_id redis
      }
      vrrp_script chk_redis {
          script "/etc/keepalived/scripts/redis_check.sh"   ###监控脚本
          interval 2                  ###监控时间
          timeout  2                                  ###超时时间
          fall     3                  ###重试次数
      }
      vrrp_instance redis {
        state BACKUP                  ###设置为MASTER
        interface eth0              ###监控网卡
        virtual_router_id 51
        priority 100              ###权重值
        authentication {
               auth_type PASS          ###加密
               auth_pass redis        ###密码
        }
        track_script {
          chk_redis              ###执行上面定义的chk_redis
        }
        virtual_ipaddress {
             192.168.0.239            ###VIP
        }
        notify_master /etc/keepalived/scripts/redis_master.sh
        notify_backup /etc/keepalived/scripts/redis_backup.sh
        notify_fault  /etc/keepalived/scripts/redis_fault.sh
        notify_stop   /etc/keepalived/scripts/redsi_stop.sh
      }
      

      也可在本地编辑好后使用scp上传到服务器。 Master-keepalived脚本redis_master.sh:

        #!/bin/bash
        REDISCLI="/usr/local/redis/bin/redis-cli"
        LOGFILE="/usr/local/redis/var/keepalived-redis-state.log"
        echo "[master]" >> $LOGFILE
        date >> $LOGFILE
        echo "Being master...." >> $LOGFILE 2>&1
        echo "Run SLAVEOF cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE  2>&1
        sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
        echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
      

      Master-keepalived脚本redis_backup.sh:

        #!/bin/bash
        REDISCLI="/usr/local/redis/bin/redis-cli"
        LOGFILE="/usr/local/redis/log/keepalived-redis-state.log"
        echo "[backup]" >> $LOGFILE
        date >> $LOGFILE
        echo "Being slave...." >> $LOGFILE 2>&1
        sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
        echo "Run SLAVEOF cmd ..." >> $LOGFILE
        $REDISCLI SLAVEOF 192.168.0.231 6379 >> $LOGFILE  2>&1
      
  • 公共脚本配置

    • redis_chekc.sh

      可以使用netcat来检测redis服务的状态.

      安装nc,使用nc检测redis的服务而不是ping
      $ wget http://sourceforge.net/projects/netcat/files/netcat/0.7.1/netcat-0.7.1.tar.gz
      $ tar -zxvf netcat-0.7.1.tar.gz
      $ ./configure
      $ make && make install

      建立脚本:

      $ vim /etc/keepalived/scripts/redis_check.sh
      

      脚本内容:

      #!/bin/bash
      ALIVE=`/usr/local/redis/bin/redis-cli PING`
      
      if ["$ALIVE" == "PONG"] ;then
            echo $ALIVE
            exit 0
      else 
            echo $ALIVE
            exit 1
      fi
      
    • redis_fault.sh

      建立脚本:

      $ vim /etc/keepalived/scripts/redis_fault.sh 
      

      脚本内容:

      # !/bin/bash
      
      LOGFILE=/usr/local/redis/var/keepalived-redis-state.log 
      echo "[faule]" >> $LOGFILE 
      date >> $LOGFILE
      
    • redis_stop.sh

      建立脚本:

      $ vim /etc/keepalived/scripts/redis_stop.sh 
      

      脚本内容:

      # !/bin/bash
      
      LOGFILE=/usr/local/redis/var/keepalived-redis-state.log 
      echo "[stop]" >> $LOGFILE 
      date >> $LOGFILE
      

测试

  • 主从复制测试

    先打开Master上的Redis,再打开Backup上的Redis,分别查看info

    $ redis-cli info  
    

    Master的rule显示master为正常,slave会显示为192.168.0.232

    在Master上写入数据,在Backup读取

    $ redis-cli set a hello  //Master 写入数据
    $ redis-cli get a        //Backup 读取数据
    "hello"
    

    主从复制正常。

  • 主从切换测试
    先后启动Master和Backup上的keepalived,在Master上查看vip

    $ ip a
    

    enter image description here

    从图中可以看出vip已经绑定在Master上,现在通过vip来进行数据测试

    $ redis-cli -h 192.168.0.239 set a test
    $ redis-cli -h 192.168.0.239 get a //测试虚拟ip数据读取
    //在主机和从机上分别进行数据读取
    $ redis-cli get a
    

    enter image description here

    从图中可以看出测试无问题。

    现在关闭Master上的redis,来看VIP能否正常漂移,并进行数据测试。

    $ killall -9 redis-server
    $ redis-cli -h 192.168.0.232 info
    

    enter image description here

    现在给redis写入数据,测试看Master恢复服务后能否正常恢复数据

    $ redis-cli -h 192.168.0.239 set a nihao
    

    恢复Master上的Redis,查看ip a

    $ service redis-server start
    $ ip a
    $ redis-cli get a
    

    enter image description here

    数据恢复,VIP切换回Master,Redis高可用环境搭建成功.

下一篇进行redis调优的测试和读写分离测试。

标签: redis, keepalived, 高可用

已有 2 条评论

  1. [...]来源来源 http://blog.molibei.com/archives/87[...]

  2. [...]来源来源 http://blog.molibei.com/archives/87[...]

添加新评论