RabbitMQ Tutorial: Creating Highly Available Message Queues Using RabbitMQ

Every day in the world of modern technology, high availability has become the key requirement of any layer in technology. Message broker software has become a significant component of most stacks. In this article, we will present a RabbitMQ tutorial: how to create highly available message queues using RabbitMQ.

RabbitMQ is an open-source message broker software (also called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ server is written in the Erlang programming language.

The RabbitMQ Cluster

Clustering connects multiple nodes to form a single logical broker. Virtual hosts, exchanges, users and permissions are mirrored across all nodes in a cluster. A client connecting to any node can see all the queues in a cluster.

Clustering enables high availability of queues and increases the throughput.

A node can be a Disc node or RAM node. RAM node keeps the message state in memory with the exception of queue contents which can reside on a disk if the queue is persistent or too big to fit into memory.

RAM nodes perform better than Disc nodes because they don’t have to write to a disk as much as disk nodes. But, it is always recommended to have disk nodes for persistent queues.

We’ll discuss how to create and convert RAM and Disk nodes later in the post.

Prerequisites:

Network connection between nodes must be reliable.

All nodes must run the same version of Erlang and RabbitMQ.

All TCP ports should be open between nodes.

We have used CentOS for the demo. Installation steps may vary for Ubuntu and OpenSuse. In this demo, we have launched two m1.small servers in AWS for master and slave nodes.

1. Install Rabbitmq

   Install Rabbitmq in master and slave nodes.
   
   

   $ yum install rabbitmq-server.noarch

2. Start Rabbitmq

   /etc/init.d/rabbitmq-server start

3. Create the Cluster

   Stop RabbitMQ in Master and slave nodes. Ensure service is stopped properly.
   
   

   /etc/init.d/rabbitmq-server stop

   
   Copy the file below to all nodes from the master. This cookie file needs to be the same across all nodes.
   
   

   $ sudo cat /var/lib/rabbitmq/.erlang.cookie

   
   Make sure you start all nodes after copying the cookie file from the master.
   
   Start RabbitMQ in master and all nodes.
   
   

   $ /etc/init.d/rabbitmq-server start

   
   Then run the following commands in all the nodes, except the master node:
   
   

   $ rabbitmqctl stop_app

   $ rabbitmqctl reset

   $ rabbitmqctl start_app

   
   Now, run the following commands in the master node:
   
   

   $ rabbitmqctl stop_app

   $ rabbitmqctl reset

   
   Do not start the app yet.
   
   The following command is executed to join the slaves to the cluster:
   
   

   $ rabbitmqctl join_cluster rabbit@slave1 rabbit@slave2

   
   Update slave1 and slave2 with the hostnames/IP address of the slave nodes. You can add as many slave nodes as needed in the cluster.
   
   Check the cluster status from any node in the cluster:
   
   

   $ rabbitmqctl cluster_status

   
   By default, the cluster stores messages on the disk. You can also choose to store Queues in Memory.
   
   You can have a node as a RAM node while attaching it to the cluster:
   
   

   $ rabbitmqctl stop_app

   $ rabbitmqctl join_cluster --ram rabbit@slave1

   
   It is recommended to have at least one disk node in the cluster so that messages are stored on a persistent disk and can avoid any loss of messages in case of a disaster.
   
   The performance of RAM nodes are a little better than disk nodes, and gives you better throughput.

4. Set the HA Policy

   The following command will sync all the queues across all nodes:
   
   

   $ rabbitmqctl set_policy ha-all "" '{"ha-mode":"all","ha-sync-mode":"automatic"}'

   
   Ex: Policy where queues whose names begin with “ha.” are mirrored to all nodes in the cluster: $ rabbitmqctl set_policy ha-all “^ha.” ‘{“ha-mode”:”all”}‘

5. Test the Queue mirror

   We are going to run a sample python program to create a sample queue. You need the below packages installed from where you want to run the program.
   
   Install python-pip
   
   

   $ yum install python-pip.noarch

   
   Install Pika
   
   

   $ sudo pip install pika==0.9.8

   
   Create send.py file and copy the content below. You need to update the “localhost” with name/ip of master/slave node.
   
   #!/usr/bin/env python
   import pika
   
   connection = pika.BlockingConnection(pika.ConnectionParameters(
           host='localhost'))
   channel = connection.channel()
   
   channel.queue_declare(queue='hello')
   
   channel.basic_publish(exchange='',
                         routing_key='hello',
                         body='Hello World!')
   print " [x] Sent 'Hello World!'"
   connection.close()
   Run the python script using the command:
   
   

   $ python send.py

   
   This will create a Queue (hello) with a message on the RabbitMQ cluster.
   
   Check if the message is available across all nodes.
   
   

   $ sudo rabbitmqctl list_queues

   
   Now, create a file named receive.py and copy the content below.
   
   

   #!/usr/bin/env python

   import pika

   connection = pika.BlockingConnection(pika.ConnectionParameters(

    'localhost'))

   channel = connection.channel()

   channel.queue_declare(queue='hello')

   def callback(ch, method, properties, body):

    print " [x] Received %r" % (body,)

   channel.basic_consume(callback,

    queue='hello',

    no_ack=True)

   print ' [*] Waiting for messages. To exit press CTRL+C'

   channel.start_consuming()

   
   Run the script and check the Queue in either the slave or master:
   
   

   $ sudo rabbitmqctl list_queues

6. Setup Load Balancer

   Now, we have multiple MQs running in a cluster. All are in sync and have the same queues.
   
   How do we point our application to the cluster? We can’t point our application to a single node. If a node fails, we need a mechanism to auto-failover to other nodes in the cluster. There are multiple ways to achieve it. But, we prefer to use the load balancer.
   
   There are two advantages in using the load balancer:
   1. High availability
   2. Better network throughput because the load is evenly distributed across nodes.

Create a load balancer in front of it and map the backend MQ instance. You can choose either HAProxy or Apache or Nginx or any hardware load balancer you use in your organization.

If the servers are running in an AWS VPC, then choose internal load balancer. Update the application to point to the load balancer endpoint.

Get Started with AWS

Receive AWS tips, DevOps best practices, news analysis, commentary and more. Sign up for our blog here and set your topic and frequency preferences. Or, download our guide on getting started with AWS, establishing a secure AWS enterprise architecture with Flux7 Landing Zones.