In my last two blog posts, I covered how to deploy a single RabbitMQ node for testing or non HA queues and RabbitMQ Cluster for HA queues.

In this blog post, I’ll cover the latest and the most reliable scenario, with a slight twist. The basic idea behind queue federation is to deal with load-balancing messages across queues on different brokers. If you have a set of queues federated with each other, then producers can publish into them and consumers can consume from them from any location. This time we’ll use Google Compute.

In this scenario we will also use a cluster of 2 nodes. We need to create a wrapper box built on top of the RabbitMQ Cluster Node to create the federated cluster. We will call it RabbitMQ Federated Node. The box wrapper has this variables:

Name Value TYPE
upstream BINDING TO RabbitMQ Cluster Node
rabbitmq RabbitMQ BOX

We need to create a post_configure Event with the following script to initialize or join an existing cluster. This is a great example of box composition, this box is adding the federation logic to the RabbitMQ Cluster Node box without having to modify the original box.

#!/bin/bash

rabbitmqctl set_parameter federation-upstream {{ instance }} '{"uri":"amqp://{{ upstream.rabbitmq.username }}:{{ upstream.rabbitmq.password }}@{{ upstream.address.private }}:5672","expires":3600000}'
rabbitmqctl set_policy --apply-to exchanges federate-me "^amq\." '{"federation-upstream-set":"all"}'

Step 1: Deploying the Cluster in GCE

Click the “New Instance” button and select the RabbitMQ Node for Cluster:

rabbitmq-federation-part-3-1.png

In the next step, you will be asked to create a deployment profile, which lets you pick the configurations for deployment such as cloud provider, datacenter, etc.

Click New Profile. Create a new deployment profile with default values.

  • Be sure to select Automatic in security groups. Using this option, Cloud Application Manager will automatically create a security group configured with the rules needed by RabbitMQ.
  • For persistent queues, make sure you select an EBS backed image and instance combination.

    rabbitmq-federation-part-3-2.png

Fill the COOKIE field with the one you want to be shared between all the RabbitMQ nodes of the cluster. We will use this convention for naming the profiles: – i.e., rabbitmq-prod-us-west-1, as the environment. Deploy it.

Step 2: Deploying the Second Node of the Cluster in GCE

  • Before launching the second node, the master must be already deployed in GCE. follow the same steps as the master, but select “RabbitMQ Federated Node” for the new instance.
  • Just prior to hitting “Deploy,” select the binding to the master node of RabbitMQ for building the cluster.
  • Deploy the second node using the same name for the environment as the zone profile you are using to deploy.
  • Repeat these steps if you need to add more nodes.

There you have it, a federated node of RabbitMQ Cluster.

This wraps up my 3-part RabbitMQ blog post series. Hope you enjoyed it. Email me if you have any questions.

Want to Learn More About Cloud Application Manager and ElasticKube?

Cloud Application Manager is a powerful, scalable platform for deploying applications into production across any cloud infrastructure – private, public or hosted. It provides interactive visualization to automate application provisioning, including configuration, deployment, scaling, updating and migration of applications in real-time. Offering two approaches to cloud orchestration — Cloud Application Manager and ElasticKube — enterprise IT and developers alike can benefit from multi-cloud flexibility.

Explore ElasticKube by visiting GitHub (curl -s https://elastickube.com | bash).

Visit the Cloud Application Manager product page to learn more.