SOA Graphic

Reading Time: about 7 minutes.

Service-oriented architecture (SOA) is a method of deploying and running software by using multiple smaller services connected over a network. This network can be local to a machine. SOA typically takes the form of virtualized containers communicating to each other over a local network bridge.

More and more, software capabilities are being delivered as services. Software running in a SOA needs to utilize these services, which may be changing locations regularly across clouds and networks as resources, are automatically provisioned or decommissioned through failure or horizontal scaling. The process of locating and connecting to services is called service discovery, and it's a growing, changing part of the SOA landscape.

Locating services involves two steps. The first step is service registration. When a new container or software capability is provisioned it needs to announce its availability via some form of registry. This registration also includes the configuration parameters needed to use the service, such as host name, port, and API versions. The second step is service discovery. This step involves clients querying the central registration to find available services that match specific requirements, such as network, cloud location, capabilities, or capacity.

In this article, we will look at different options for container clustering and discovery using Docker.

Docker: The Lightweight Container

Docker has become a very popular method to distribute and deploy Linux applications. It is a lightweight container system that wraps an application in a complete file system containing everything that it needs to function. One of the biggest benefits of this approach is that distributed software will always run with the same dependencies, configuration, and environment across any host system, whether the host system is bare metal, virtual server, or cloud.

While Docker containers provide services that should work in a SOA, they don't automatically publish and broadcast the services they provide. In an environment that may automatically scale containers horizontally as they are needed, this can be painstaking, and sometimes even a headache. So let's take a look at some of the services and solutions that solve the issue.

Discovery Backends

A discovery backend is a service that registers other services. Usually, it consists of some sort of replicated key/value database with multiple nodes. The redundancy provided by replication means that the discovery backend is always available. In fact, a SOA with service discovery needs to have a readily available discovery backend. Here are a few options for discovery backend services.

Etcd

Etcd is distributed and maintained as part of the CoreOS project. It is billed as "a distributed, reliable key-value store for the most critical data of a distributed system." It expects to be part of a cluster of several nodes, but can run as a standalone service as well. Typically, Etcd containers are small. For example, the Microbox Docker image for Etcd is only 17MB.

One nice element of Etcd is that it is a RESTful service. Once the Docker image is running, you can test the installation using curl or other command-line clients. Check-out this article by Nitish Tiwari for details. Note: When deploying servers to host Etcd, you will need to open up ports 4001 and 7001 in your firewall.

Consul

Consul is a more sophisticated discovery backend than Etcd in that it provides not only a RESTful key/value store, but a DNS-based service registry. The DNS interface can allow some applications to deploy to a SOA architecture with little or no code change. Instead of needing to add RESTful key/value lookups to find services, for instance, an application can use a DNS query.

Consul also provides built-in support for SOA clusters operating across multiple data centers. This is essential in certain hybrid cloud infrastructures and geographically distributed systems. The official Docker Consul image is very well-documented, but because of its additional features, it requires a little more work to setup. Note: When deploying servers to host Consul, keep ports 8300 through 8600 open. Also, remember that DNS can be a UDP protocol. So plan your firewall accordingly.

Service Discovery with Docker Swarm

There is no single route to container discovery, even when you're just looking at a limited set of tools. Many methods for service discovery with Docker use Docker Swarm. This is a native clustering solution for Docker that uses a collection of machines to emulate a single virtual Docker host server. It uses the standard Docker remote API, which allows any tool that normally communicates with Docker to use the Swarm installation to perform standard tasks. The recommended Swarm configuration requires five different virtual servers, consisting of two swarm management servers, two nodes, and a Consul server.

The Docker Swarm documentation includes a tutorial for deploying an implementation. It uses Consul as the discovery backend and can easily be adapted to deploy on CenturyLink Compute servers. Note: A quick-and-easy Docker Swarm tutorial is also available on GitHub Gist. This implementation uses an Etcd discovery backend and only requires two virtual servers.

Deploying CenturyLink Compute Servers for Docker Swarm

To deploy Docker Swarm, you will need multiple virtual servers. While it is possible to configure and provision them manually one at a time using the CenturyLink Cloud Control Portal, the approach is more time consuming than need be. Instead, it is faster to use an automation tool like Ansible to handle provisioning and configuration for us.

In this tutorial, we will use Ansible to provision five virtual servers that can be used to host Docker Swarm.

Tools Used

CenturyLink Cloud Compute servers are high-performance cloud servers. These enterprise-grade virtual machines are easy to deploy and manage from the CenturyLink Cloud Console automatically via our powerful API.

Ansible is a tool for configuring and managing clouds and servers. It handles software deployment, cloud resource provisioning, task execution, and configuration management.

Before We Start

If you don’t have a CenturyLink Cloud account yet, head over to our website and sign up for a free trial. You’ll need it to access CenturyLink Cloud Compute servers. Because automatically deploying and configuring virtual servers can be a complicated topic, you might want to check-out our comprehensive collection of knowledge base articles on virtual servers.

You will also want to install Ansible on your machine. Most Linux distributions have Ansible packages available. For example, on Ubuntu or Debian you would use the following command:

sudo apt-get install ansible

For other distributions and operating systems, follow the installation instructions from the Ansible documentation.

Using Ansible to Provision Servers Automatically

To create an Ansible playbook for automatically provisioning and deploying CenturyLink Cloud Compute servers, follow these instructions:

  1. With a text editor, create a file called deploy_docker_swarm.yml and edit the file to look like the following example. Pay close attention to the task definition. Check the Ansible clc_server module documentation for more options.

    You can change any of the settings in the vars: section to customize your deployment. Be sure to change the value of server_password to the password to be used for your new virtual servers.

    ---
    - name: Deploy Ubuntu hosts at CLC
      hosts: localhost
      vars:
        server_group: 'Default Group'
        server_count: 5
        server_password: 'PASSWORD'
      gather_facts: False
      connection: local
      tasks:
        - name: Deploy servers
          clc_server:
            name: "swarm"
            template: ubuntu-16-64
            count: "{{ server_count }}"
            group: "{{ server_group }}"
            password: "{{ server_password }}"
          register: clc
        - name: debug
          debug: var=clc.server_ids
    
  2. Execute the playbook using the following command. Replace <your-ctl-username> with your CenturyLink Cloud username, and <your-ctl-password> with your CenturyLink Cloud password. It can take a little while to provision multiple servers.

    $ CLC_V2_API_USERNAME='<your-ctl-username>' CLC_V2_API_PASSWD='<your-ctl-password>' ansible-playbook -vv deploy_docker_swarm.yml
    

Note: To further automate deployment using Ansible, check-out our tutorial on automatically deploying virtual servers using CenturyLink Runner.

Next Steps

Once you have provisioned a collection of servers, deploy a SOA discovery backend using the tutorial in the Docker Swarm manual. Automatic service discovery is still a developing field, and dozens of different success stories and configurations are available on the web. Still, with this tutorial you are on the road to having automatic horizontal scaling and container discovery for your Docker-backed applications.

Docker offers many options to deploy and manage software on CenturyLink Cloud Compute servers. Check-out our other articles about Docker for more ideas and inspiration.

Sign-up for our Developer-focused newsletter CODE. Designed hands-on by developers, for developers. Keep up to date on topics of interest: tutorials, tips and tricks, and community building events.

We’re a different kind of cloud provider – let us show you why.