Deployment of a High-Availability Cluster on FreeBSD


Authors : Aurélien Bompard & Grégory Bernard - with the contribution of Farzad Farid.

The goal of this document is to describe precisely the various steps involved in the deployment and configuration of a Linux-HA cluster on FreeBSD using the ports. We will focus on a precise description of the various steps involved to achieve the deployment of a working solution.

This documentation is based on FreeBSD 6.2.p5 and heartbeat-1.2.5_3

Description of a FreeBSD HA cluster


Choose the right hardware:


Choosing the right hardware is certainly not the most simple thing to do. In order to have the best results possible you should try to follow these rules :
For this example we have choosen a hardware that will have a double attachement (dedicated Ethernet + Serial link).
Once you have your hardware delivered, here comes the "fun" part of the process.

Install Heartbeat from the ports tree:

# cd /usr/ports/sysutils/heartbeat/
# make install clean

Check the right detection of the serial port: # dmesg | grep ^sio
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A

The serial port must be correctly detected.

Test serial connectivity:

On the first machine: # cat < /dev/cuad1
On the seconde machine: # echo "hello" > /dev/cuad1
Result on the first machine: # cat < /dev/cuad1
hello

Serial connectivity is OK !

Test ethernet connectivity:

Both nodes of the cluster should have 2 dedicated Ethernet interfaces (2 physicaly separated eth). One will be on the network where the services will be available, the other one will be dedicated for the cluster (for testing purposes).

Configure Heartbeat:

Copy the default heartbeat configurations: # cd /usr/local/etc/ha.d
# cp /usr/local/share/doc/heartbeat/ha.cf .
# cp /usr/local/share/doc/heartbeat/haresources .
# cp /usr/local/share/doc/heartbeat/authkeys .

Edit authkeys: # cat /usr/local/etc/ha.d/authkeys
auth 1
1 sha1 YourSecretClusterPassword

Fix permissions on authkeys: # chmod 600 /usr/local/etc/ha.d/authkeys

Edit ha.cf: # grep -v "^#" /usr/local/etc/ha.d/ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
warntime 5
initdead 60
udpport 694
baud 9600
serial /dev/cuad1
bcast em1
auto_failback off
node node1.your-cluster.tld
node node2.your-cluster.tld
debug 1

For more information on the available options, please refer to http://linux-ha.org/GettingStarted and http://linux-ha.org/ha.cf

Configure a first shared resource: # echo "node1.your-domain.tld 192.168.1.1" >> /usr/local/etc/ha.d/haresources
Where "192.168.1.1" is your shared IP address.

The element of each line in this file is the name of the master node for the following resources. Depending on your choice on ``auto_failback``, the resources will or will not be moved back to this node when it is back up.

Resources are actually scripts that will be called with "start" as an argument when the cluster switches from a node to the other. For more information on the way to configure resources in heartbeat, please refer to http://linux-ha.org/GettingStarted and http://linux-ha.org/haresources.

Activate Heartbeat: # echo 'heartbeat_enable="YES"' >> /etc/rc.conf
Start Heartbeat: # /usr/local/etc/rc.d/heartbeat.sh start
Check the logs: # tail -f /var/log/ha-log
In 30 seconds, you'll see Heartbeat take the shared IP address.
Check it with ``ifconfig``.

Check the cluster's failover system:

Do the same on the second machine. The ``haresources`` file **must** be exactly the same on both machines.

Start heartbeat on the second machine, and watch the logs. # tail -f /var/log/ha-log
Then, on the first machine, ask heartbeat to pass the resources (the shared IP) to the second machine.

With the following command: # /usr/local/lib/heartbeat/hb_standby
You'll see hearbeat taking up the resources on the second machine.
Check it with ``ifconfig``.

Additional configuration:

Install additional services and add them to ``haresources`` (all on the same line). You may use the provided "Service" script which will call FreeBSD init scripts in /usr/local/etc/rc.d and in /etc/rc.d.

If you need the "Filesystem" script, you'll need to edit it because it is still very linux-specific. You'll also need to install the "sysutils/fuser" port. The patch we used is provided too. In general, always test the resource scripts you are going to use, because they may have been designed with Linux in mind.

Some files included with the present distribution are buggy:

In particular the files included in /usr/local/etc/ha.d/resource.d/