Managing logs for cloud infrastructure can be a daunting task. Fortunately, we have tools like Graylog to help us out. Graylog is useful for storing, searching, and analyzing logs from a multitude of different sources. One common log source is a Tomcat app running in a Docker container. However, getting logs from containers isn't always simple. Oftentimes, you have to spend extra time configuring other services like syslog or logstash to deal with log aggregation.
Thankfully, there is a faster way. The most recent release of Docker, v1.8.0, makes collecting and sending logs to Graylog much easier. Additionally, the logs are significantly enhanced with extra data such as container ID, container command, and configurable tags. These extra fields can be invaluable when troubleshooting an application problem. In this tutorial I will cover how to set up Docker to natively send Tomcat logs to Graylog. Using the ideas explained in this tutorial should help simplify your existing Tomcat container deployments and logging requirements. As an added bonus, the techniques discussed here can also be generalized to other services running in Docker.
Docker version: >=1.8.0
Install Docker. Docker's native support for Graylog Extended Log Format (GELF) is new, and you will need to install the latest version of Docker for your host OS. For detailed instructions, please consult Docker Supported Installations. I'm using Ubuntu vivid and in order to install the latest version I simply run the following command:
$ curl -sSL https://get.docker.com/ | sh
Create a Java8 Dockerfile. We'll be using Dockerfiles to create two Docker images. The first is strictly Java8. In a later step, we'll use this image to build the Tomcat8 server. I like to create separate directories for each Dockerfile in case any helper scripts are needed during the image creation. First
mkdir jdk8-oracleand then
cd jdk8-oracle. Place the contents below into a file called
Dockerfile. The Dockerfiles in this tutorial are available from GitHub.
FROM ubuntu:vivid MAINTAINER Matthew Close "https://github.com/mclose" RUN apt-get -y install software-properties-common RUN add-apt-repository ppa:webupd8team/java RUN apt-get update && apt-get -y upgrade # accept oracle license RUN echo debconf shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | /usr/bin/debconf-set-selections # install oracle java 8 RUN apt-get -y install oracle-java8-installer && apt-get clean RUN update-alternatives --display java ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
Build the image. This step will take a few minutes to complete if you haven't already pulled down the Ubuntu vivid container image.
$ docker build -t mclose/jdk8-oracle .
Create a Tomcat8 image. Similar to above, first
cd tomcat8. Then place the contents below into a
Dockerfile. In order to use the latest version of Tomcat8, I'm installing tomcat manually as opposed to using
apt-getwhich will install an older version. Basically, this Dockerfile creates a Tomcat user, downloads and installs the latest version of Tomcat8, and finally runs Tomcat. In order to keep the container running we
tail -Fthe output of the logs. More importantly, by using the
tailcommand, Tomcat logs are sent to stdout. The GELF log driver then picks up both stdout and stderr and sends them to Graylog. By tailing the catalina.log files we are able to direct Tomcat logs to Graylog. To generalize this to other applications besides Tomcat is simply a matter of tailing your service's log file.
FROM mclose/jdk8-oracle MAINTAINER Matthew Close "https://github.com/mclose" RUN adduser --system --shell /bin/bash --gecos 'Tomcat user' --group --disabled-password --home /home/tomcat tomcat RUN wget http://www.us.apache.org/dist/tomcat/tomcat-8/v8.0.24/bin/apache-tomcat-8.0.24.tar.gz -O /tmp/tomcat.tar.gz RUN mkdir -p /usr/share/tomcat8 RUN tar xzvf /tmp/tomcat.tar.gz -C /usr/share/tomcat8 RUN rm /tmp/tomcat.tar.gz RUN ln -s /usr/share/tomcat8/apache-tomcat-8.0.24 /usr/share/tomcat RUN chown -R tomcat:tomcat /usr/share/tomcat8 CMD /bin/su - tomcat -c /usr/share/tomcat/bin/startup.sh && tail -F /usr/share/tomcat/logs/catalina.out
Build the image. This build should be much quicker than the previous one.
$ docker build -t mclose/tomcat8 .
Start a Graylog container. For the purpose of this tutorial, I'm going to use a Docker container to run Graylog. Since Docker is already installed this is the fastest way to test. However, if you already have a Graylog server configured, please feel free to use it. First
runthe container. In this example, I'm exposing two ports. Port 9000 is used for Graylog administration. Port 12201/udp is used to receive GELF log entries. For more information, see Graylog and Docker.
$ docker run -t -p 9000:9000 -p 12201:12201/udp graylog2/allinone
Login into Graylog. http://localhost:9000. For the Docker container, the user is
adminand the password is
Create the GELF UDP input in Graylog. From the main screen select System -> Inputs. In the dropdown on the next screen, select GELF UDP and click Launch New Input.
Configure the GELF UDP input in Graylog. Simply give the input a Title and click the Launch button at the bottom of the window. There is no need to change any of the defaults for this tutorial.
Verify the Graylog input is running. Once launched, the input should now show as running in Graylog.
Start a Tomcat8 container. There are a couple of important things at work in the
dockercommand below: First
-p 8080:8080exposes the default Tomcat port so we can validate it is running. Next
--log-driver=gelfselects GELF for logs,
--log-opt gelf-address=udp://localhost:12201sets the destination of the logs, and
--log-opt gelf-tag="tomcat8 example"will set a useful tag in the Graylog entries. If you are using your own server, change localhost to the correct IP. For a complete list of Docker GELF options, please see Docker's Configuring Logging Drivers.
$ docker run -t -p 8080:8080 --log-driver=gelf --log-opt gelf-address=udp://localhost:12201 --log-opt gelf-tag="tomcat8 example" mclose/tomcat8
Verify Tomcat is running. Tomcat should have successfully started during the previous step. To verify go to http://localhost:8080.
View Tomcat logs in Graylog. You should now see log entries for Tomcat within Graylog.
View log detail. Click on one of the entires and you can see the amount of extra detail you get from the GELF driver. Not only can you see the log entry, but now it includes a wealth of extra data about the Docker container that sent the log.
I hope this tutorial has convinced you that using the new Docker GELF drivers is well worth it. The extra data about Docker containers included in every log will help reduce troubleshooting time. Additionally, your Tomcat deployments should now be easier.