Setting up a failover between two webservers using Keepalived on CentOS 6 and 7

Introduction:

Floating IP addresses are an important part of setting up a High Availability (HA) infrastructure, as the name already suggests its advantage is that it can be re-assigned to different servers in case of a failover. 

Pre-requisites:

For this tutorial we'll need the following software:

- Keepalived v1.2.22 (06/17/2016). You can download the latest package from http://www.keepalived.org/download.html
- Two servers running either CentOS 6 or 7. (IP's should be in the same network range).
- IPv4 failover address (Instructions on how to obtain such an address can be found on https://support.tilaa.com/hc/en-us/articles/228650467

- Asynchronous routing should then be enabled by adding the following to /etc/sysctl.conf :

         net.ipv4.conf.default.rp_filter = 0 
         net.ipv4.conf.all.rp_filter = 0

Installation

Step 1. Download & Install Keepalived
The most convenient and easiest solution is to install Keepalived using the package manager. To do so please run the following command:

yum install keepalived

Alternatively, you can install it from source using the instructions below:

yum install gcc kernel-headers openssl-devel
wget http://www.keepalived.org/software/keepalived-1.2.22.tar.gz
tar -zxvf keepalived-1.2.22.tar.gz
cd keepalived-1.2.22
./configure --prefix=/; make; make install
chkconfig keepalived on

Step 2. Configure Keepalived 
Keepalived saves its configuration file in: /etc/keepalived/keepalived.conf You can use the below example configuration on webserver1 (MASTER): 

vrrp_script chk_httpd {
        script "pidof httpd"
        interval 2
}
vrrp_instance VI_1 {
        # The interface keepalived will manage
        interface eth0
        state MASTER
        # How often to send out VRRP advertisements
        advert_int 2
        # The virtual router id number to assign the routers to
        virtual_router_id 51
        # The priority to assign to this device. This controls
        # who will become the MASTER and BACKUP for a given
        # VRRP instance (a lower number get's less priority)
        priority 100
        authentication {
                auth_type PASS
                auth_pass SimplePassword
        }
        unicast_src_ip primary_ip_of_web1
        unicast_peer {
                primary_ip_of_web2
        }
        track_script {
                chk_httpd
        }
        # The virtual IP addresses to float between nodes.
        virtual_ipaddress {
                failover_ipv4_address
        }
}

The following configuration should be placed on webserver2 (BACKUP):

vrrp_script chk_httpd {
        script "pidof httpd"
        interval 2
}
vrrp_instance VI_1 {
        # The interface keepalived will manage
        interface eth0
        state BACKUP
        # How often to send out VRRP advertisements
	advert_int 2
        # The virtual router id number to assign the routers to
        virtual_router_id 51
        # The priority to assign to this device. This controls
        # who will become the MASTER and BACKUP for a given
        # VRRP instance (a lower number get's less priority).
        priority 50
        authentication {
                auth_type PASS
                auth_pass SimplePassword
        }
	unicast_src_ip primary_ip_of_web2
	unicast_peer {
		primary_ip_of_web1
	}
        track_script {
                chk_httpd
        }
        # The virtual IP addresses to float between nodes.
        virtual_ipaddress {
                failover_ipv4_address
        }
}

Verify your setup!

To verify your setup is working we can check the following:

- Use tcpdump to verify if VRRP traffic is exchanged between the two nodes.
- Check your server logs
- Verify if the VIP is assigned to your network interface
- Firewall exceptions

1. Using tcpdump to check for VRRP traffic:
If you've followed this tutorial then you can use the following command in order to verify if VRRP packets are being exchanged:
sudo tcpdump -n -v -i eth0 vrrp

 2. Inspect the server logs:
Try starting keepalived with the -d flag, this will dump your configuration to your server log. It won't report errors but you'll get a clear overview of the setup. In addition during a failover you'll notice that the backup host will accept it's new MASTER state, these messages are typically found in /var/log/messages.

3. Verify if the VIP is assigned correctly:
You can use the following command in order to verify if the failover IP has been assigned correctly: ip a s dev eth0 You should see the failover IP mentioned in the output.

4. Firewall exceptions:
VRRP uses protocol number 112, so make sure ingoing/outbound traffic is allowed. For IPTables you can use the rules mentioned below, alternatively, you may specify a source and destination address using the -s and -d flags respectively:
iptables -I INPUT -p vrrp -j ACCEPT
iptables -I OUTPUT -p vrrp -j ACCEPT

Tips & Tricks / References:

Obviously, the above-mentioned configuration is just a basic example but it offers a basic starting point to go from. Keepalived offers a lot of additional information and instructions making it very flexible in its use. I strongly encourage to keep reading more documentation, there're several sources available:

http://www.keepalived.org/documentation.html <-- Documentation is deprecated but can still be quite useful.
https://github.com/acassen/keepalived <- Github repository