一、  简介:

LVSLinux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。

目前有三种IP负载均衡技术(VS/NATVS/TUNVS/DR);

八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。

Virtual Server via Network Address Translation(VS/NAT)

通过网络地址转换,调度器重写请求报文的目标地址,根据预设的调度算法,将请求分派给后端的真实服务器;真实服务器的响应报文通过调度器时,报文的源地址被重写,再返回给客户,完成整个负载调度过程。

Virtual Server via IP Tunneling(VS/TUN)

采用NAT技术时,由于请求和响应报文都必须经过调 度器地址重写,当客户请求越来越多时,调度器的处理能力将成为瓶颈。为了解决这个问题,调度器把请求报文通过IP隧道转发至真实服务器,而真实服务器将响 应直接返回给客户,所以调度器只处理请求报文。由于一般网络服务应答比请求报文大许多,采用 VS/TUN技术后,集群系统的最大吞吐量可以提高10倍。

Virtual Server via Direct Routing(VS/DR)

VS/DR通过改写请求报文的MAC地址,将请求 发送到真实服务器,而真实服务器将响应直接返回给客户。同VS/TUN技术一样,VS/DR技术可极大地提高集群系统的伸缩性。这种方法没有IP隧道的开 销,对集群中的真实服务器也没有必须支持IP隧道协议的要求,但是要求调度器与真实服务器都有一块网卡连在同一物理网段上。

 

二、  环境:

Director Server master真实IP                   192.168.4.181

      DirectorServer backup真实IP         192.168.4.182
     Director ServerVIP                                192.168. 4.185
      RealServer1(WEB_A)IP                         192.168. 4.183
      RealServer1(WEB_B)IP                         192.168. 4.184

 

三、  安装配置基于DR模式Lvs集群:

1.从官方网站下载ipvsadmkeepalive

下载地址:

下载ipvsadm-1.24.tar.gz,因为最新版ipvsadm-1.25这个版本只支持2.6.28以上的内核,RHEL 5.4内核版本达不到要求,所以选择安装前一个版本。

下载地址:

下载keepalived-1.1.20.tar.gz

 

2、安装ipvsadmmaster backup

1)创建一个连接文件,其命令为:

[root@master ~]# ln -s/usr/src/kernels/2.6.18-238.el5-i686 /usr/src/linux

注意一定要与当前的运行的内核相一致,因为/usr/src/kernels目录下可能有多个目录。如果不创建这个连接文件,在编译时会出错,从而不能继续进行安装。

[root@master ~]# ll

总计 336

-rw------- 1 root root   1666 08-23 19:08 anaconda-ks.cfg

-rw-r--r-- 1 root root  29211 08-23 19:08 install.log

-rw-r--r-- 1 root root   4582 08-23 19:07 install.log.syslog

-rw-r--r-- 1 root root  36598 08-26 13:43 ipvsadm-1.24.tar.gz

-rw-r--r-- 1 root root 233002 08-26 13:44keepalived-1.1.20.tar.gz

[root@master ruanjian]# tar -zxvfipvsadm-1.24.tar.gz

[root@master ruanjian]# cd ipvsadm-1.24

[root@master ipvsadm-1.24]# make

[root@master ipvsadm-1.24]# make install

 

2)检验ipvsadm是否被正确安装

(1)执行ipvsadm,看是否有如下输出。

[root@master ipvsadm-1.24]# ipvsadm

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

 

(2)检查当前加载的内核模块,看是否存在ip_vs模块。

[root@master ipvsadm-1.24]# lsmod | grepip_vs

ip_vs                  78081  0

 

1:只有执行ipvsadm以后,才会在内核加载ip_vs模块。

2:不能以查进程的方式判断ipvs是否运行。

3:如果下载最新的ipvsadm-1.25.tar.gz这个版本,在创建连接文件/usr/src/linux后,执行编译时,可能需要修改/boot/grub/grub.conf启动内核名称。一旦当前运行内核与连接文件所代表的内核名不一致时,将出现找不到*.h这样的错误,从而导致安装不能正常进行。

 

3、安装keepalived.

在负载均衡服务器上执行masterbackup

1、解压
[root@master ~]# tar -zxvf keepalived-1.1.20.tar.gz

[root@master ~]# cd keepalived-1.1.20

[root@master keepalived-1.1.20]#./configure --prefix=/usr/local/keepalived

[root@master keepalived-1.1.20]# make

[root@master keepalived-1.1.20]# makeinstall

[root@master keepalived-1.1.20]# cp/usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/

[root@master keepalived-1.1.20]# chmod +x/etc/init.d/keepalived

 

 [root@masterkeepalived-1.1.20]# cp /usr/local/keepalived/etc/sysconfig/keepalived

/etc/sysconfig/

[root@master keepalived-1.1.20]# mkdir/etc/keepalived

[root@master keepalived-1.1.20]# cp/usr/local/keepalived/etc/keepalived/keepalived.conf

                                                                 /etc/keepalived/

[root@master keepalived-1.1.20]# cp/usr/local/keepalived/sbin/keepalived /usr/sbin/

[root@master keepalived-1.1.20]# servicekeepalived start

启动 keepalived                                          [确定]

4、开启负载服务器路由机制(masterbackup

[root@master keepalived-1.1.20]# echo"1" > /proc/sys/net/ipv4/ip_forward

[root@master keepalived-1.1.20]# sysctl –p           使配置生效 显示下面信息

net.ipv4.ip_forward = 0

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route =0

kernel.sysrq = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.shmmax = 4294967295

kernel.shmall = 268435456 

 

 

5、建立负载服务器启动脚本(masterbackup

#vi /sbin/lvsdr.sh
#!/bin/bash

VIP=192.168.4.185

RIP1=192.168.4.183

RIP2=192.168.4.184

/sbin/rc.d/init.d/functions

case "$1" in

start)

      echo "start LVS of DirectorServer"

      #Set the Virtual IP Address

      /sbin/ifconfig eth0:1 $VIP broadcast $VIP netmask 255.255.255.255 up

      /sbin/route add -host $VIP dev eth0:1

      #Clear IPVS Table

      /sbin/ipvsadm -C

      #Set Lvs

      /sbin/ipvsadm -A -t $VIP:80 -s wrr

      /sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g

      /sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g

      #Run Lvs

      /sbin/ipvsadm

;;

stop)

echo "Close LVS Directorserver"

/sbin/ifconfig eth0:1 down

/sbin/ipvsadm -C

;;

*)

echo "Usage0{start|stop}"

exit 1

esac

 

 6、分配权限(master backup

[root@master keepalived-1.1.20]# chmod 755 /sbin/lvsdr.sh

7、执行测试(master backup

[root@master keepalived-1.1.20]# /sbin/lvsdr.sh start

start LVS of Director Server

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  ->RemoteAddress:Port           ForwardWeight ActiveConn InActConn

TCP 192.168.4.185:http wrr

 -> 192.168.4.184:http          Route   1      0         0        

 -> 192.168.4.183:http          Route   1      0         0

  查看ifconfig是否有ifcfg-eth0:0(有就对了)

[root@master keepalived-1.1.20]# ifconfig

eth0     Link encap:Ethernet  HWaddr00:0C:29:A1:73:5E 

         inet addr:192.168.4.181 Bcast:192.168.4.255 Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST MTU:1500  Metric:1

          RX packets:7001 errors:0 dropped:0overruns:0 frame:0

         TX packets:7851 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:3747687 (3.5 MiB)  TXbytes:786537 (768.1 KiB)

         Interrupt:59 Base address:0x2000

 

eth0:0   Link encap:Ethernet  HWaddr00:0C:29:A1:73:5E 

         inet addr:192.168.4.185 Bcast:192.168.4.185 Mask:255.255.255.255

         UP BROADCAST RUNNING MULTICAST MTU:1500  Metric:1

         Interrupt:59 Base address:0x2000

  查看route -n 路由表是否多了eth0:0路由(有就对了)

[root@master keepalived-1.1.20]# route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

192.168.4.185   0.0.0.0         255.255.255.255 UH    0     0        0 eth0

192.168.4.0     0.0.0.0         255.255.255.0   U    0      0        0 eth0

169.254.0.0     0.0.0.0         255.255.0.0     U    0      0        0 eth0

0.0.0.0         192.168.4.1     0.0.0.0         UG   0      0        0 eth0

# /sbin/lvsdr.sh stop
  查看ifconfig是否有ifcfg-eth0:0 (无就对了)
  查看route -n 路由表是否多了eth0:0路由(无就对了)
# /sbin/lvsdr.sh adsa
  是否提示参数错误,只能使用{start|stop}

8、配置后端WEB服务器

192.168.4.183192.168.4.184上分别建立如下脚本。
[root@WEB_A ~]# vim /sbin/realdr.sh

      #!/bin/bash

      VIP=192.168.4.185

      /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up

      /sbin/route add -host $VIP dev lo:0

      echo "1">/proc/sys/net/ipv4/conf/default/arp_ignore

      echo "2">/proc/sys/net/ipv4/conf/default/arp_announce

      echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore

      echo "2">/proc/sys/net/ipv4/conf/all/arp_announce

      sysctl –p

9、配置权限

[root@WEB_A ~]# chmod 755 /sbin/realdr.sh

10、在两台web服务器上分别执行其指命。

/sbin/realdr.sh start

 

11、在masterbackup上面

[root@WEB_A ~]# vim /etc/keepalived/keepalived.conf

文件内容

! Configuration File for keepalived

global_defs {

 notification_email {

   zhangxl@pcikchina.com

  }

 notification_email_from xwluan@tsong.cn

 smtp_server mail.pcikchina.com

 smtp_connect_timeout 30

 router_id LVS_DEVEL

}

vrrp_instance VI_1 {

  state MASTER  //备份服务器设置为backup

  interface eth0

  virtual_router_id 51

  priority 100  备份服务器设置小于100

  advert_int 3

  authentication {

      auth_type PASS

      auth_pass 1111

   }

  virtual_ipaddress {

      192.168.4.185

   }

}

virtual_server 192.168.4.185 80 {

  delay_loop 6  //6秒查询

  lb_algo wrr   //lvs算法

  lb_kind DR    //(Direct Route)

  persistence_timeout 60  //同一IP的连接60秒内被分配到同一台realserver

  inhibit_on_failure  //web挂掉的时候,前面请求的用户,可以继续打开网页,但是后面的请求不会调度到挂掉的web上面。

  protocol TCP   //TCP协议检查realserver状态

  real_server 192.168.4.183 80 {

      weight 3    //权重

      TCP_CHECK {

          connect_timeout 10   //10秒无响应超时

          nb_get_retry 3

         connect_port 80

          delay_before_retry 3

      }

   }

  real_server 192.168.4.184 80 {

      weight 3

      TCP_CHECK {

          connect_timeout 10

          nb_get_retry 3

         connect_port 80

          delay_before_retry 3

      }

   }

}

 

backup上面

! Configuration File for keepalived

global_defs {

 notification_email {

   zhangxl@pcikchina.com

  }

 notification_email_from xwluan@tsong.cn

 smtp_server mail.pcikchina.com

 smtp_connect_timeout 30

 router_id LVS_DEVEL

}

vrrp_instance VI_1 {

  state BACKUP  //备份服务器设置为backup

  interface eth0

  virtual_router_id 51

  priority 80  备份服务器设置小于100

  advert_int 3

  authentication {

      auth_type PASS

      auth_pass 1111

   }

  virtual_ipaddress {

      192.168.4.185

   }

}

virtual_server 192.168.4.185 80 {

  delay_loop 6  //6秒查询

  lb_algo wrr   //lvs算法

  lb_kind DR    //(Direct Route)

  persistence_timeout 60  //同一IP的连接60秒内被分配到同一台realserver

  inhibit_on_failure  //web挂掉的时候,前面请求的用户,可以继续打开网页,但是后面的请求不会调度到挂掉的web上面。

  protocol TCP   //TCP协议检查realserver状态

  real_server 192.168.4.183 80 {

      weight 3    //权重

      TCP_CHECK {

          connect_timeout 10   //10秒无响应超时

          nb_get_retry 3

         connect_port 80

          delay_before_retry 3

      }

   }

  real_server 192.168.4.184 80 {

      weight 3

      TCP_CHECK {

          connect_timeout 10

          nb_get_retry 3

         connect_port 80

          delay_before_retry 3

      }

   }

}

 

12,启动keepalived

[root@master keepalived-1.1.20]# /etc/init.d/keepalived start

13:设置成自启动

[root@master keepalived-1.1.20]# vim /etc/rc.local  //里面添加

/etc/init.d/keepalived restart

/etc/lvsdr.sh start

 

 

 

 

四、  安装配置基于隧道模式Lvs集群:

1、安装ipvsadmin

和上面一样,在此略过

2、创建LVS  TUN模式启动脚本(masterbackup

[root@linux-centos ~]# vim/etc/init.d/lvsdstun

#!/bin/bash

VIP=192.168.4.185

RIP1=192.168.4.183

RIP2=192.168.4.184

/etc/rc.d/init.d/functions

case "$1" in

start)

          echo "start LVS of DirectorServer Tun"

          /sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up

          /sbin/route add -host $VIP dev tunl0

          /sbin/ipvsadm -C

          /sbin/ipvsadm -A -t $VIP:80 -s rr

          /sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -i

          /sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -i

          /sbin/ipvsadm

       ;;

stop)

          echo "stop LVS of DirectorServer Tun"

          /sbin/ifconfig tunl0 down

          /sbin/ipvsadm -C

          ;;

*)

         echo "Usage: $0 {start|stop}"

         exit 1

esac

 

[root@linux-centos ~]# chmod +x/etc/init.d/lvsdstun

[root@linux-centos ~]# /etc/init.d/lvsdstunstart

start LVS of DirectorServer Tun

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

TCP 192.168.4.185:http rr

 -> 192.168.4.184:http          Tunnel  1      0         0        

 -> 192.168.4.183:http          Tunnel  1      0         0  

[root@linux-centos ~]# ifconfig

eth0     Link encap:Ethernet  HWaddr00:0C:29:AF:F0:E8 

         inet addr:192.168.4.182 Bcast:192.168.4.255 Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST MTU:1500  Metric:1

         RX packets:2832 errors:0 dropped:0 overruns:0 frame:0

         TX packets:1040 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:242958 (237.2 KiB)  TXbytes:105068 (102.6 KiB)

         Interrupt:59 Base address:0x2000

 

lo       Link encap:Local Loopback 

         inet addr:127.0.0.1 Mask:255.0.0.0

         UP LOOPBACK RUNNING MTU:16436  Metric:1

         RX packets:32 errors:0 dropped:0 overruns:0 frame:0

         TX packets:32 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:5821 (5.6 KiB)  TXbytes:5821 (5.6 KiB)

 

tunl0    Link encap:IPIP Tunnel HWaddr  

         inet addr:192.168.4.185 Mask:255.255.255.255

         UP RUNNING NOARP  MTU:1480  Metric:1

         RX packets:0 errors:0 dropped:0 overruns:0 frame:0

         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:0 (0.0 b)  TX bytes:0(0.0 b)

 

3、安装keepalived master backup

和上面一样,在此略过

4、WEB_A WEB_B上创建启动脚本

[root@linux-centos ~]# vim/etc/init.d/lvsrstun

#!/bin/bash

VIP=192.168.4.185

/etc/rc.d/init.d/functions

case "$1" in

start)

          echo "start LVS of RealServer Tun"

          /sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up

          /sbin/route add -host $VIP dev tunl0

          echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore

          echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce

          echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

          echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

      ;;

stop)

          echo "close LVS of RealServer Tun"

          /sbin/ifconfig tunl0 down

          echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore

          echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_announce

          echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

          echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

          ;;

*)

         echo "Usage: $0 {start|stop}"

         exit 1

esac

exit 0

 

[root@linux-centos ~]# chmod +x/etc/init.d/lvsrstun

[root@linux-centos ~]# /etc/init.d/lvsrstunstart

 

 

5.测试