Creating a Docker container in order to execute jobs to other containers

Creating a Docker container in order to execute jobs to other containers

19 May 2017 |  Devops

Last week, I wanted to implement several scripts to extract some data, generate Json files and export them to a remote server. These scripts needed to be scheduled in a crontab to be run on a daily basis. My application is split into components, following micro-services architecture patterns and each component runs in a Docker container.

Extracting and manipulating data is not a big deal, nevertheless, there were a few rules I wanted to follow. First, components [MB1] are not responsible for extracting data when I need them, they have to offer a way to do it. This means I don’t want to modify the code base of an app. Secondly, I don’t want to store configuration on a server that cannot be versioned in GIT. Finally, I needed to use the container environment variables when running command in the crontab.

After spending some time designing the best solution, I decided to create a Docker container that will run all my scripts and send all reports to a remote server. Obviously, I am isolated from the inside of a container and I don’t have access to the host machine. I cannot run any script on the host but I can run command on another container following the DooD principle. The basic idea of DooD or Docker outside of Docker, is to access the host’s Docker installation from within the container. Basically, we link the Docker socket from the host to the crontab container. I have implemented this in the past when playing with continuous integration systems that rely on Docker to build applications based on pull-requests

You can have a look at my Github repository. If you run the container, it should display a “hello world message every two minutes.

Running a crontab container

First thing, create a Dockerfile which will store your container provisioning information. Update the apk repository and install Docker. This is mandatory to run a Docker command inside the container. Another solution could consist of sharing the Docker binary in between the host and the container.

Dockerfile

FROM gliderlabs/alpine:3.4
MAINTAINER "Julien Dubreuil"
RUN apk update && apk add --no-cache docker

Next, create a file called crontab which will contain a standard crontab file with your tasks. In my example, I’m printing a greeting using an environment variable. When the echo command is executed, the shell will get the value from the environment variable defined when starting the container.

crontab

  */2 * * * * echo "Hello ${ENV_NAME}" >> /var/log/cron/cron.log 2>&1
  # An empty line is required at the end of this file for a valid cron file.

Create a file called run-crond.sh that will be responsible for placing our crontab file at a good place on the server when starting the container. By default, Docker’s environment variables is not passed to any cron command. A workaround consists to inject all of them inside the crontab file at the beginning.

The first line of the following chunk of code will add all variables prefixed with ENV_ to the crontab file.

run-crond.sh

  #!/bin/sh
  env | egrep '^ENV_' | cat - /tmp/crontab > /etc/crontabs/root
  crond -L /var/log/cron/cron.log "$@" && tail -f /var/log/cron/cron.log

Finally, update your Dockerfile to copy the crontab file [MB2] on the container and execute the run_crond.sh script when starting the container.

Dockerfile

  FROM gliderlabs/alpine:3.4
  MAINTAINER "Julien Dubreuil"
  RUN apk update && apk add --no-cache docker
  COPY crontab /tmp/crontab
  COPY run-crond.sh /run-crond.sh
  RUN chmod -v +x /run-crond.sh
  RUN mkdir -p /var/log/cron && touch /var/log/cron/cron.log
  CMD ["/run-crond.sh"]

Now you are all set, you can build an image and create a new container.

  $ docker build -t docker-cron-ooc-example .
  $ docker run -it
  -e ENV_NAME=f00 \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  --name my-docker-cron-ooc \
  $ docker-cron-ooc-example

After a few minutes, you should see logs and your greeting messages.

  Hello f00
  crond: USER root pid  12 cmd echo "Hello ${ENV_NAME}" >> /var/log/cron/cron.log 2>&1
  Hello f00
  crond: USER root pid  13 cmd echo "Hello ${ENV_NAME}" >> /var/log/cron/cron.log 2>&1
  Hello f00
  crond: USER root pid  14 cmd echo "Hello ${ENV_NAME}" >> /var/log/cron/cron.log 2>&1
Julien Dubreuil

Vous avez une idée, un projet web à réaliser ?

Ensemble, mettons en oeuvre sa réussite. Je vous accompagne dans vos projets, depuis l'élaboration du cahier des charges jusqu'à la mise en production. Pour plus d'information n'hésitez pas à me contacter.

Contactez-moi