The goal of this tutorial is to set up continuous and automatic deployment of an application with Jenkins on a Docker architecture. Continuous and automatic deployment is one of the key concepts of the DevOps movement. The purpose is to foster application industrialization by enabling a system to: test the application, build the container image, and deploy it to production.
Prerequisites:
Docker
Docker-compose
How It Works
Our deployment system will use 4 layers to deploy the application to production:
- Gogs: A private git project hosting platform (presented as a web interface very similar to GitHub). Gogs can be replaced by any git server, as we will only use it as a repository.
- Jenkins: Jenkins will use a script called “Jenkinsfile” to build the application container image.
- Private Docker Registry: Will contain the container images internal to our system.
- WatchTower: A worker that regularly checks for newer images of deployed containers.
To trigger a deployment, simply push code to Gogs. Gogs will send an alert to Jenkins through a webhook. Jenkins will then start a “pipeline” build – a script divided into several stages (contained in the Jenkinsfile). We will create three stages: the first builds the project’s Docker image (using a Dockerfile), the second launches a temporary container with the image from the previous stage to run unit tests, and the final stage pushes the image to our private Docker registry. The last step is to wait for WatchTower to detect the image change and replace the old container with an updated one.
Setting Up the Docker Architecture
We will use docker-compose to set up the required architecture.
We need to create a Dockerfile (placed at “/home/user/jenkins/Dockerfile”) to create an image of a modified Jenkins container with Docker API access:
FROM jenkins
MAINTAINER Flavien JOURDREN <[email protected]>
USER root
RUN apt-get update -qq
RUN apt-get install -qqy apt-transport-https ca-certificates
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
RUN echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list
RUN apt-get update -qq
RUN apt-get install -qqy docker-engine
RUN usermod -aG docker jenkins
USER jenkins
docker-compose.yml contents:
version: '2'
services:
mysql:
image: mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=aCoolPassword
- MYSQL_DATABASE=gogs
- MYSQL_USER=root
- MYSQL_PASSWORD=aCoolPassword
volumes:
- "/containersVolume/database:/var/lib/mysql"
gogs:
image: gogs/gogs
restart: always
links:
- "mysql:db"
ports:
- "10022:22"
- "10080:3000"
volumes:
- "/containersVolume/gogs:/data"
jenkins:
build: /home/user/jenkins
restart: always
ports:
- "8080:8080"
- "50000:50000"
volumes:
- "/containersVolume/jenkins:/var/jenkins_home"
- "/var/run/docker.sock:/var/run/docker.sock"
registry:
image: registry
restart: always
ports:
- "127.0.0.1:5000:5000"
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
volumes:
- "/containersVolume/registry:/data"
watchtower:
image: v2tec/watchtower
restart: always
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
command: --interval 30 --cleanup
application:
image: localhost:5000/application
restart: always
Running the docker-compose.yml file:
docker-compose up -d --build
Configuring Gogs
Gogs is an open source project developed in Go whose goal is to create a GitHub clone. The benefit is being able to host git projects on a server internal to our system. Later, Gogs will also allow us to set up webhooks to automatically trigger Jenkins builds on git push.
After installing Gogs, create the private git project that will contain the application source code.
Configuring Jenkins
Jenkins will require at least the following plugins: Gogs, Git, Docker, and Docker Pipeline.
To retrieve source code from Gogs, we need to configure “credentials” – an element that tells Jenkins how to connect to an external service. Here, Jenkins will connect with our Gogs account. Credential configuration: Global credentials, Username with password.
Create a Jenkins Pipeline project, add the git project URL, and associate the credential.
Configuring the Webhook
The webhook allows triggering a Jenkins “build” when a push occurs on the git project. Go to Gogs in the project settings, then “webhook”. Insert the Jenkins hook URL: http://<Jenkins server>:8080/<jenkins project name>-webhook/
Configuring WatchTower
WatchTower is an open source project that automatically updates Docker container images. It works with a worker that checks at regular intervals whether a newer image exists in the associated Docker registry.
All WatchTower configuration options are available on the GitHub page: https://github.com/v2tec/watchtower
In reality, the processing interval and image cleanup are already configured in the docker-compose:
command: --interval 30 --cleanup
Creating the Jenkinsfile and Dockerfile
The Jenkinsfile is executed by Jenkins during the pipeline run. In our case, it creates a container to execute our application’s Dockerfile, run tests, and push the image to the Docker registry.
The Dockerfile automates Docker image creation. It installs all dependencies needed for the application and the application itself. A Dockerfile can also inherit from another Dockerfile, allowing it to build on community-created scripts.
The Jenkinsfile and Dockerfile are also pushed to the git repository at the project root.
The example covers deploying a simple JavaScript API.
Jenkinsfile:
node {
def app
currentBuild.result = "SUCCESS"
try {
stage('Clone repository') {
checkout scm
}
stage('Build image') {
app = docker.build("test")
}
stage('Test image'){
app.inside {
echo 'test'
}
}
stage('Push image') {
docker.withRegistry('https://127.0.0.1:5000') {
app.push("latest")
}
}
}
catch (err) {
currentBuild.result = "FAILURE"
throw err
}
}
Dockerfile:
FROM node
MAINTAINER Flavien JOURDREN <[email protected]>
ADD . /home/node/app
WORKDIR /home/node/app
RUN npm install
CMD npm run start
EXPOSE 3000
What the Dockerfile does:
- “FROM node”: Inherits from the community-created “node” image on Docker Hub.
- “ADD . /home/node/app”: Adds the application files to the /home/node/app directory in the container.
- “RUN npm install”: Installs npm dependencies.
- “CMD npm run start”: Starts the application each time a container from this image is started.
- “EXPOSE 3000”: Exposes the container’s port 3000.
Final Setup and Testing
- Push the source code to deploy to the “production” branch of the remote git repository.
- Verify on the Jenkins web interface that the build was triggered through the webhook.
- Once the application build is complete, verify the image was pushed to the Docker registry by querying its API (check:
http://registryIP:5000/v2/_catalog). - Start containers with docker-compose.
- Verify deployed containers are working correctly.
- Push a new version to the “production” branch.
- This time, WatchTower will detect the changes once the image build is complete and updated on the Docker registry, and will update the containers of that image.
- Test the latest application changes.