One of the biggest advantages of working on a team with a DevOps culture is the ability to quickly react to the needs of the business. If demand for a particular piece of tooling or automation is identified, a solution can be rapidly delivered to end users. By quickly creating automation for common operational tasks, a team can greatly improve their productivity. While this is great for quickly responding to a business’s needs, it is easy for teams to become overwhelmed with the amount of tools they have available to them. Worse yet, what if you are on a cross-functional team with a variation in technical skill-sets? It can be challenging to create solutions for a common denominator, especially with the short development times that can accompany projects being rapidly released. Having to dig through documentation to relearn a tool you used three months ago can be a huge time sink. Ironically, tools that were meant to increase productivity can sometimes reduce it.

ChatOps

One of the ways DevOps teams have solved these problems is through a concept called ChatOps. Originally coined by the GitHub team, ChatOps is a way to standardize and organize a library containing multiple tools and simple documentation into a chat client interface. By consolidating a user’s interaction with tooling into a format many people are comfortable with (i.e.: chat clients), teams can quickly leverage a wide array of tools. In addition to the benefits provided by the standardization and organization of a team’s tooling, exposing your toolset in a chat client is also a great way to promote collaboration and education for available functionality. By integrating ChatOps with clients that have the concept of team channels, such as Slack, teams can see how their colleagues are using available tools and what they are working on. As a Devops shop ourselves, the team at CenturyLink Cloud is no stranger to the rapid development of software. Many of our engineering teams have adopted ChatOps tools to automate their workflows. Things like monitoring, deploying, and reporting have been automated and exposed by our Slack instance.

Being on the Customer Excellence team at CenturyLink Cloud, I was often quickly developing reporting tools, mostly via PowerShell scripts, that would take some simple input from a user, query our APIs for data, then parse everything together and export a user-friendly report. Having a wide customer base with diverse reporting needs, I found myself managing a repository of dozens of tools. To make matters worse, we had to frequently support a sales force that sometimes wasn’t technologically comfortable with having to run a script or command line tool to generate a piece of information they needed. Instead of burning a large amount of time developing intricate front end interfaces for our customer reporting tools, I decided to follow the example set by our engineering teams and use ChatOps to solve our problems.

Jarvis

Meet Jarvis. Jarvis is a Slackbot that has functionality built on top of GitHub’s Hubot to enable anyone in our organization to quickly retrieve information about any of our customers.

For example, what if I wanted to know how many servers a particular customer has running?

Jarvis1.png

Or what data centers a customer has a footprint in?

Jarvis2.png

Maybe I just want a quick snapshot of a customer’s environment.

Jarvis3.png

Users are able to quickly access a list of functionality via Hubot’s built in help command.

Jarvis4.png

Hubot

Hubot is written in CoffeeScript on Node.js. Since most of the tools I was developing for the team were reports created using data retrieved from the CenturyLink Cloud API using PowerShell, I either needed to redevelop all of the tools I created from scratch, or find a way to kick off and retrieve data from PowerShell scripts from within Node.js. Luckily, there is a resource called PoshHubot created by Matthew Hodgkins that facilitates this exact functionality.

After installing PushHubot by following the documentation on GitHub, developing new functionality requires a PowerShell script and an accompanying CoffeeScript file. Let’s use Jarvis’s server count functionality as an example. The PowerShell code to retrieve the data from the CenturyLink Cloud API looks like this:

First we have to declare the function, parameters and a hash table to store a response:

function serverCount
{
    [CmdletBinding()]
    Param
    (
        [Parameter(Mandatory=$true)]
        $alias
    )

    $result = @{}

Next we need to log in to the CenturyLink Cloud APIs. I do this via functions within a PowerShell module I made in order to reuse the same code in all of Jarvis’s scripts:

  $global:session = loginCLCAPIV1

        $HeaderValue = loginCLCAPIV2

In order to get a customer’s server count across each data center they use, we need to call the API for the list of locations the customer has access to:

        $DCURL = "https://api.ctl.io/v2/datacenters/$alias"
        $datacenterList = Invoke-RestMethod -Uri $DCURL -ContentType "Application/JSON" -Headers $HeaderValue -Method Get
        $datacenterList = $datacenterList.id

Now we can iterate through our data center list and query the API for the customer’s servers in each location:

$allServers = @()

        Foreach ($i in $datacenterList)
        {
            $JSON = @{AccountAlias = $alias; Location = $i} | ConvertTo-Json
            $response = Invoke-RestMethod -uri "https://api.ctl.io/REST/Server/GetAllServersForAccountHierarchy/" -ContentType "Application/JSON" -Method Post -WebSession $session -Body $JSON -errorAction stop
            $allServers += $response.AccountServers
        }

Finally, we can send a successful result back to Hubot, or a message if we were unsuccessful. In order for things to look correct in the eventual Slack output, we need to follow Slack’s formatting guide with our output:

$result.output = "Alias $($alias) has *$($serverCount)* servers`."

        $result.success = $true
    }
    catch
    {
      $result.output = "Failed to return account info for $($alias)."
      $result.success = $false
    }
    return $result | ConvertTo-Json
}

Now we need to create the accompanying CoffeeScript file. We start by including Edge.js which allows Node.js to call PowerShell, and add instructions for Edge to call Powershell:

edge = require("edge")

# Build the PowerShell that will execute
executePowerShell = edge.func('ps', -> ###
  . .\scripts\serverCount.ps1
  serverCount -alias $inputFromJS.aliasName
###
)

After that, we can do a regex capture to store a specific Slack message in a variable, which will trigger the Hubot listener and can be stored into a variable to be sent to PowerShell:

module.exports = (robot) ->
  # Capture the account alias being requested
  robot.respond /server count (.*)/i, (msg) ->
    # Set the requested alias to a variable
    aliasName = msg.match[1]
    msg.send "Getting the server count for all accounts rolling up to the parent account alias #{aliasName}."
    # Build an object to send to PowerShell
    psObject = {
      aliasName: aliasName
    }

Next, we build the PowerShell callback:

    callPowerShell = (psObject, msg) ->
      executePowerShell psObject, (error,result) ->
        if error
          msg.send ":fire: An error was thrown in Node.js/CoffeeScript"
          msg.send error
        else
          result = JSON.parse result[0]

One last thing, we need to add some logic to notify the user whether or not the script was run successfully:

if result.success is true
            msg.send ":white_check_mark: #{result.output}"
          else
            msg.send ":warning: #{result.output}"

Finally, we can call PowerShell:

callPowerShell psObject, msg

By following the PoshHubot framework I laid above, we are able to create powerful functionality with the CenturyLink Cloud API, such as reporting on customer utilization.

Jarvis5.png

If you would like to learn more about Hubot, you can read Matthew Hodgkin’s guide about PoshHubot. If you’re curious about Jarvis, you can see all of the functionality we have developed in this GitHub repository.

Get started on CenturyLink Cloud with a free trial!