Initially, when we deployed multiple data centers we used a unique host for each one. We always knew that approach was going to be a temporary solution. Still, it was a quick and easy way to get isolated cluster environments up and running. However, we recently deployed an API fix that allows us to route all traffic internally rather than requiring the client to configure where it wants to go. The beauty is that you can query our Orchestrate API using an API key for any data center or any app. Internally, we route the query to the right destination via NGINX.

DevOps Benefit

There's no need to configure data center-specific URLs for your application. We've done the heavy lifting. You simply query the Orchestrate API and it works. You don't have to worry about where your data is located. That means you can remove all host-specific code that you'd been required to add before. The Orchestrate API makes your life easier. You offload the job of figuring out the best way to get your data to you quickly. You might say it's a return to using the client with default settings.

GSLB Diagram

Routing with NGINX

The main challenge in routing with NGINX is that NGINX doesn’t actually know where the right data center is located -- at least initially when it receives a request. It has no way of knowing that information until the request reaches our back-end systems.

Here's how it works. The configuration queries the local back-ends, assuming that it will be a local query. Each back-end can determine very quickly if the key is actually for a different data center. If so, it returns a 401 error containing an error message with the right data center details in it. It also adds a special header: X-Accel-Redirect. The header tells NGINX to reprocess the query with a new URI, which we use to determine where to send the request.

A simplified version of our NGINX configuration looks like this:

server {
  listen 80
  location ~* ^/proxy/(.*)$ {
    internal;
    proxy_pass https://$1$request_uri;
  }
  location / {
    proxy_pass http://localhost:8000;
  }
}

X-Accel-Redirect Header

The internal location is used only if the X-Accel-Redirect header is set; so the first pass always routes to http:localhost:8000. If that returns the header, the request is reprocessed (discarding the response from the first query) and the request ends up proxied to the host set in the header.

This bit of coding is nice as it allows our back-ends to continue processing queries and not hang on to connections longer than necessary. It also reuses our existing security_unauthorized 401 error message that contained the URL of the data center serving the application. Thus, it required little significant changes on the back-end. All of this works really well assuming that the client is routing to the closest data center to them. For that we use a technology called Global Service Load Balancing (GSLB).

So, what exactly is GSLB?

GSLB is a technique enabling intelligent traffic distribution that allows a single domain name to be served from any number of different geographical locations. When a user tries to go to https://api.orchestrate.io we're able to evaluate exactly where the client is located geographically and then route them to the closest data center we have. This feature is used by virtually all major web properties and is already well-documented.

There are several providers offering GSLB like DynDNS and Amazon Route 53. These products provide general case solutions and tend to work in broad strokes. They segment the world into fairly large zones and treat everybody in each of the zones equally. This approach is fine if you are a typical web service with a presence on either US coast and perhaps a data center in Europe.

What's Unique about Orchestrate GSLB?

Our use case is quite a bit different than the typical GSLB offering. The traditional segmentation approach doesn’t work for us because we have multiple data centers in each region. Our customers care a great deal about latency issues associated with accidentally routing a request to the wrong data center.

Bind with Views

We use bind with views that specifically segment out the entire IP space and route each IP to the closest data center. Since we know the IPs of the data centers we can ensure that local clients are always routed to the closest front-ends every time. If the client calls us from another location, we use geographic IP information in order to route the client to what is likely the closest data center. If it's not the closest one, then NGINX routes it to the proper location.

Summary

Orchestrate API brings you truly intelligent API traffic routing. Our solution eliminates the need to configure data center-specific URLs for your applications, regardless of where you are or where our global data centers are located. That means instant global service load balancing with low-latency. Looking to try this out for yourself? Start with a free trial of Orchestrate.

Our lines of code, our technology -- is your platform. We’ll never stop improving and iterating new products and services to better your business.