CenturyLink is sponsoring an exciting initiative created by ECG Robotics this year, providing cloud computing resources to host websites for teams participating in FIRST Robotics, a family of international competitive robotics programs.

ECG Robotics has nearly 100 members who are high school students from Greensboro and Guilford County, North Carolina. Its five teams compete in two levels of FIRST’s “sport of the mind,” where teams build robots to score points on a challenge field. With one program entering the last few weeks of the build season, we checked in with Aidan Hunt, who told us that his team’s robot should soon be ready for competitions. Meanwhile, one of ECG’s teams in the other program brought home a Connect Award at their recent qualifier.

Today, let’s take a closer look at the team’s project to provide shared hosting services for their fellow teams, and the technology and applications they’re using to host and support these sites.

Servers

The team exclusively uses CenturyLink's Cloud products. Websites are hosted on a production Bare Metal server, speced out with a quad-core Intel Xeon processor and 8 GB of RAM. An additional virtual server is used to host mail services and other internal sites such as a status page. With the flexible CenturyLink Cloud Control Portal, the team can quickly spin up additional lightweight servers for testing without disrupting production. Unlike many competing platforms, CenturyLink Cloud allows users to customize CPU, RAM, and storage independently, rather than forcing users into predefined packages. The team uses Ubuntu 18.04.1 LTS, providing a nice balance of recent tool versions and the stable package quality needed for production.

Software Stack

The team follows the configuration-as-code approach by keeping all config files in a Git repo privately stored on GitHub. This approach removes many of the manual tasks involved in infrastructure provisioning, maintenance and software configuration, and also allows the team to roll back potentially disruptive changes.

Technology for the operating system, content management, and traffic management are mostly open source, including the WordPress content management system that the project's users work with to build their sites. Though the users won't ever encounter them, two open-source projects are especially critical to the platform's success:

  • Docker. The team uses Docker and Docker Compose to define and run multi-container Docker applications, and to handle starts and stops of internal and client workloads. With Docker, containers can use their own base image, run predefined setup commands, and export specific ports and data paths to the rest of the system. Docker Compose allows multiple containers to be combined, environment variables to be inserted, and virtual networks to be automatically created. It also allows containers to use their names instead of IP addresses to connect to each other.
  • Traefik. Traefik is a "reverse proxy" that ensures the smooth flow of network traffic between clients and servers. One feature the team appreciates is that it reads what hosts it is supposed to proxy traffic to from Docker labels at runtime, removing the need to touch config files or restart it to add or rename sites. Traefik supports TLS and HTTP/2, even when the actual applications it proxies to don't.

These programs work together to implement the microservices architecture, so that subsystems, such as the MySQL databases required by WordPress, are stored in their own containers, rather than being shared between teams. This arrangement allows Docker to be used to manage almost all activity on the service, and to control file storage and access for effective backup and security. The team has previous experience with Docker from developing their StrangeScout competition scouting program, which they will also be hosting for other teams on the server.

Security

From the start, the platform was designed with high security in mind. The Traefik container handles all traffic on ports 80 and 443, and enforces the use of HTTPS for all connections to the outside. Traefik automatically requests the certificates it needs from Let's Encrypt, the nonprofit certificate authority, so full encryption of all traffic comes at no cost to the team. Additionally, Traefik makes it easy to set the HSTS header, which blocks protocol downgrade attacks.

On the application side, the microservices design pattern provides further benefits. Giving users the ability to host their own dynamic websites can be a challenge for security, but the team provides their platform with several lines of defense. First, the PHP-FPM and web server processes run as non-root users within the container. Secondly, the container is only given access to data on a filesystem mounted with the noexec option on the virtual server, further protecting against compromise. Lastly, Docker networks are configured to limit unneeded access; for example, the MySQL containers can only be accessed by their respective WordPress install.

Backups

Handling client data also requires a carefully defined backup plan. The team uses CenturyLink Object Storage with restic for easy, reliable backups of config files and webroots. It’s capable of backing up local files to a number of different backend repositories: a local directory, an SFTP server or an S3-compatible object storage service. The team had previous experience using restic with S3, so it was easy to upgrade to CenturyLink’s lower-cost Object Storage backend.

Final Thoughts

Designing a production system requires consideration of many factors and assembling a technical stack involving many different programs working together. Hunt says the team appreciates CenturyLink's investment not only in their project, but in providing them the ability to get real-world skills in system development and administration, communications, and technical support by working on a real project while in high school. Here at CenturyLink, we're thrilled to give back to the FIRST community by providing the infrastructure for this exciting program.

FIRST®, FIRST® Robotics Competition, and FIRST® Tech Challenge are registered trademarks of FIRST® www.firstinspires.org which is not overseeing, involved with, or responsible for this activity, product, or service.