一、 简介:
LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。
目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);
八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。
Virtual Server via Network Address Translation(VS/NAT)通过网络地址转换,调度器重写请求报文的目标地址,根据预设的调度算法,将请求分派给后端的真实服务器;真实服务器的响应报文通过调度器时,报文的源地址被重写,再返回给客户,完成整个负载调度过程。
Virtual Server via IP Tunneling(VS/TUN)采用NAT技术时,由于请求和响应报文都必须经过调 度器地址重写,当客户请求越来越多时,调度器的处理能力将成为瓶颈。为了解决这个问题,调度器把请求报文通过IP隧道转发至真实服务器,而真实服务器将响 应直接返回给客户,所以调度器只处理请求报文。由于一般网络服务应答比请求报文大许多,采用 VS/TUN技术后,集群系统的最大吞吐量可以提高10倍。
Virtual Server via Direct Routing(VS/DR)VS/DR通过改写请求报文的MAC地址,将请求 发送到真实服务器,而真实服务器将响应直接返回给客户。同VS/TUN技术一样,VS/DR技术可极大地提高集群系统的伸缩性。这种方法没有IP隧道的开 销,对集群中的真实服务器也没有必须支持IP隧道协议的要求,但是要求调度器与真实服务器都有一块网卡连在同一物理网段上。
二、 环境:
Director Server master真实IP 192.168.4.181
DirectorServer backup真实IP 192.168.4.182 Director ServerVIP 192.168. 4.185 RealServer1(WEB_A)IP 192.168. 4.183 RealServer1(WEB_B)IP 192.168. 4.184
三、 安装配置基于DR模式Lvs集群:
1.从官方网站下载ipvsadm、keepalive:
下载地址:
下载ipvsadm-1.24.tar.gz,因为最新版ipvsadm-1.25这个版本只支持2.6.28以上的内核,RHEL 5.4内核版本达不到要求,所以选择安装前一个版本。
下载地址:
下载keepalived-1.1.20.tar.gz
2、安装ipvsadm(master backup)
(1)创建一个连接文件,其命令为:
[root@master ~]# ln -s/usr/src/kernels/2.6.18-238.el5-i686 /usr/src/linux
注意一定要与当前的运行的内核相一致,因为/usr/src/kernels目录下可能有多个目录。如果不创建这个连接文件,在编译时会出错,从而不能继续进行安装。
[root@master ~]# ll
总计 336
-rw------- 1 root root 1666 08-23 19:08 anaconda-ks.cfg
-rw-r--r-- 1 root root 29211 08-23 19:08 install.log
-rw-r--r-- 1 root root 4582 08-23 19:07 install.log.syslog
-rw-r--r-- 1 root root 36598 08-26 13:43 ipvsadm-1.24.tar.gz
-rw-r--r-- 1 root root 233002 08-26 13:44keepalived-1.1.20.tar.gz
[root@master ruanjian]# tar -zxvfipvsadm-1.24.tar.gz
[root@master ruanjian]# cd ipvsadm-1.24
[root@master ipvsadm-1.24]# make
[root@master ipvsadm-1.24]# make install
(2)检验ipvsadm是否被正确安装
(1)执行ipvsadm,看是否有如下输出。
[root@master ipvsadm-1.24]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
(2)检查当前加载的内核模块,看是否存在ip_vs模块。
[root@master ipvsadm-1.24]# lsmod | grepip_vs
ip_vs 78081 0
注1:只有执行ipvsadm以后,才会在内核加载ip_vs模块。
注2:不能以查进程的方式判断ipvs是否运行。
注3:如果下载最新的ipvsadm-1.25.tar.gz这个版本,在创建连接文件/usr/src/linux后,执行编译时,可能需要修改/boot/grub/grub.conf启动内核名称。一旦当前运行内核与连接文件所代表的内核名不一致时,将出现找不到*.h这样的错误,从而导致安装不能正常进行。
3、安装keepalived.
在负载均衡服务器上执行master和backup
1、解压[root@master ~]# tar -zxvf keepalived-1.1.20.tar.gz[root@master ~]# cd keepalived-1.1.20
[root@master keepalived-1.1.20]#./configure --prefix=/usr/local/keepalived
[root@master keepalived-1.1.20]# make
[root@master keepalived-1.1.20]# makeinstall
[root@master keepalived-1.1.20]# cp/usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@master keepalived-1.1.20]# chmod +x/etc/init.d/keepalived
[root@masterkeepalived-1.1.20]# cp /usr/local/keepalived/etc/sysconfig/keepalived
/etc/sysconfig/
[root@master keepalived-1.1.20]# mkdir/etc/keepalived
[root@master keepalived-1.1.20]# cp/usr/local/keepalived/etc/keepalived/keepalived.conf
/etc/keepalived/
[root@master keepalived-1.1.20]# cp/usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@master keepalived-1.1.20]# servicekeepalived start
启动 keepalived: [确定]
4、开启负载服务器路由机制(master和backup)
[root@master keepalived-1.1.20]# echo"1" > /proc/sys/net/ipv4/ip_forward
[root@master keepalived-1.1.20]# sysctl –p 使配置生效 显示下面信息
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route =0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
5、建立负载服务器启动脚本(master和backup)
#vi /sbin/lvsdr.sh#!/bin/bashVIP=192.168.4.185
RIP1=192.168.4.183
RIP2=192.168.4.184
/sbin/rc.d/init.d/functions
case "$1" in
start)
echo "start LVS of DirectorServer"
#Set the Virtual IP Address
/sbin/ifconfig eth0:1 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev eth0:1
#Clear IPVS Table
/sbin/ipvsadm -C
#Set Lvs
/sbin/ipvsadm -A -t $VIP:80 -s wrr
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
#Run Lvs
/sbin/ipvsadm
;;
stop)
echo "Close LVS Directorserver"
/sbin/ifconfig eth0:1 down
/sbin/ipvsadm -C
;;
*)
echo "Usage0{start|stop}"
exit 1
esac
6、分配权限(master 和backup)
[root@master keepalived-1.1.20]# chmod 755 /sbin/lvsdr.sh7、执行测试(master 和backup)
[root@master keepalived-1.1.20]# /sbin/lvsdr.sh startstart LVS of Director Server
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
->RemoteAddress:Port ForwardWeight ActiveConn InActConn
TCP 192.168.4.185:http wrr
-> 192.168.4.184:http Route 1 0 0
-> 192.168.4.183:http Route 1 0 0
查看ifconfig是否有ifcfg-eth0:0(有就对了)[root@master keepalived-1.1.20]# ifconfig
eth0 Link encap:Ethernet HWaddr00:0C:29:A1:73:5E
inet addr:192.168.4.181 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7001 errors:0 dropped:0overruns:0 frame:0
TX packets:7851 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3747687 (3.5 MiB) TXbytes:786537 (768.1 KiB)
Interrupt:59 Base address:0x2000
eth0:0 Link encap:Ethernet HWaddr00:0C:29:A1:73:5E
inet addr:192.168.4.185 Bcast:192.168.4.185 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:59 Base address:0x2000
查看route -n 路由表是否多了eth0:0路由(有就对了)[root@master keepalived-1.1.20]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.4.185 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 192.168.4.1 0.0.0.0 UG 0 0 0 eth0
# /sbin/lvsdr.sh stop 查看ifconfig是否有ifcfg-eth0:0 (无就对了) 查看route -n 路由表是否多了eth0:0路由(无就对了)# /sbin/lvsdr.sh adsa 是否提示参数错误,只能使用{start|stop}8、配置后端WEB服务器
在192.168.4.183和192.168.4.184上分别建立如下脚本。[root@WEB_A ~]# vim /sbin/realdr.sh#!/bin/bash
VIP=192.168.4.185
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
echo "1">/proc/sys/net/ipv4/conf/default/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/default/arp_announce
echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
sysctl –p
9、配置权限
[root@WEB_A ~]# chmod 755 /sbin/realdr.sh10、在两台web服务器上分别执行其指命。
/sbin/realdr.sh start
11、在master和backup上面
[root@WEB_A ~]# vim /etc/keepalived/keepalived.conf
文件内容
! Configuration File for keepalived
global_defs {
notification_email {
zhangxl@pcikchina.com
}
notification_email_from xwluan@tsong.cn
smtp_server mail.pcikchina.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER //备份服务器设置为backup
interface eth0
virtual_router_id 51
priority 100 备份服务器设置小于100
advert_int 3
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.4.185
}
}
virtual_server 192.168.4.185 80 {
delay_loop 6 //隔6秒查询
lb_algo wrr //lvs算法
lb_kind DR //(Direct Route)
persistence_timeout 60 //同一IP的连接60秒内被分配到同一台realserver
inhibit_on_failure //当web挂掉的时候,前面请求的用户,可以继续打开网页,但是后面的请求不会调度到挂掉的web上面。
protocol TCP //用TCP协议检查realserver状态
real_server 192.168.4.183 80 {
weight 3 //权重
TCP_CHECK {
connect_timeout 10 //10秒无响应超时
nb_get_retry 3
connect_port 80
delay_before_retry 3
}
}
real_server 192.168.4.184 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
connect_port 80
delay_before_retry 3
}
}
}
在backup上面
! Configuration File for keepalived
global_defs {
notification_email {
zhangxl@pcikchina.com
}
notification_email_from xwluan@tsong.cn
smtp_server mail.pcikchina.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP //备份服务器设置为backup
interface eth0
virtual_router_id 51
priority 80 备份服务器设置小于100
advert_int 3
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.4.185
}
}
virtual_server 192.168.4.185 80 {
delay_loop 6 //隔6秒查询
lb_algo wrr //lvs算法
lb_kind DR //(Direct Route)
persistence_timeout 60 //同一IP的连接60秒内被分配到同一台realserver
inhibit_on_failure //当web挂掉的时候,前面请求的用户,可以继续打开网页,但是后面的请求不会调度到挂掉的web上面。
protocol TCP //用TCP协议检查realserver状态
real_server 192.168.4.183 80 {
weight 3 //权重
TCP_CHECK {
connect_timeout 10 //10秒无响应超时
nb_get_retry 3
connect_port 80
delay_before_retry 3
}
}
real_server 192.168.4.184 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
connect_port 80
delay_before_retry 3
}
}
}
12,启动keepalived
[root@master keepalived-1.1.20]# /etc/init.d/keepalived start13:设置成自启动
[root@master keepalived-1.1.20]# vim /etc/rc.local //里面添加/etc/init.d/keepalived restart
/etc/lvsdr.sh start
四、 安装配置基于隧道模式Lvs集群:
1、安装ipvsadmin:
和上面一样,在此略过
2、创建LVS TUN模式启动脚本(master和backup)
[root@linux-centos ~]# vim/etc/init.d/lvsdstun
#!/bin/bash
VIP=192.168.4.185
RIP1=192.168.4.183
RIP2=192.168.4.184
/etc/rc.d/init.d/functions
case "$1" in
start)
echo "start LVS of DirectorServer Tun"
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev tunl0
/sbin/ipvsadm -C
/sbin/ipvsadm -A -t $VIP:80 -s rr
/sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -i
/sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -i
/sbin/ipvsadm
;;
stop)
echo "stop LVS of DirectorServer Tun"
/sbin/ifconfig tunl0 down
/sbin/ipvsadm -C
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
[root@linux-centos ~]# chmod +x/etc/init.d/lvsdstun
[root@linux-centos ~]# /etc/init.d/lvsdstunstart
start LVS of DirectorServer Tun
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.4.185:http rr
-> 192.168.4.184:http Tunnel 1 0 0
-> 192.168.4.183:http Tunnel 1 0 0
[root@linux-centos ~]# ifconfig
eth0 Link encap:Ethernet HWaddr00:0C:29:AF:F0:E8
inet addr:192.168.4.182 Bcast:192.168.4.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2832 errors:0 dropped:0 overruns:0 frame:0
TX packets:1040 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:242958 (237.2 KiB) TXbytes:105068 (102.6 KiB)
Interrupt:59 Base address:0x2000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:32 errors:0 dropped:0 overruns:0 frame:0
TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5821 (5.6 KiB) TXbytes:5821 (5.6 KiB)
tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.4.185 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0(0.0 b)
3、安装keepalived (master 和backup)
和上面一样,在此略过
4、在WEB_A和 WEB_B上创建启动脚本
[root@linux-centos ~]# vim/etc/init.d/lvsrstun
#!/bin/bash
VIP=192.168.4.185
/etc/rc.d/init.d/functions
case "$1" in
start)
echo "start LVS of RealServer Tun"
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev tunl0
echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
echo "close LVS of RealServer Tun"
/sbin/ifconfig tunl0 down
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
[root@linux-centos ~]# chmod +x/etc/init.d/lvsrstun
[root@linux-centos ~]# /etc/init.d/lvsrstunstart