How to use Gitlab CI/CD to build, publish and deploy Docker Images

So you have your own Dockerfile and deploy your services using docker-compose? Right now you are probably just copying the needed files to your server and then deploy them using docker-compose or something similar. In this tutorial I am going to show you how you can automate all these steps with a simple .gitlab-ci.yml file using gitlab's Continuous Integration / Continuous Deployment Pipeline, which is just a fancy phrasing for a server building your code every time you commit to your git repository (or triggered by other conditions).

TL;DR

paulgoio/templates
gitlab mirror: templates to build and deploy docker images with docker-compose - paulgoio/templates

What you need to do:

  • Make an account at gitlab.com and setup a repository for the service. (Or self-host your own Gitlab instance and setup your own shared runner)
  • Make an account at hub.docker.com and make an repository with the same name as the Gitlab repository. (Or any other provider for docker repository's)
  • You should also already have a running setup for your service with a docker-compose.yml file and if you build your own container with a Dockerfile

After you have your Git and Docker repository as well as your docker-compose.yml and (Optional) your Dockerfile you can start setting up your CI/CD Pipeline.

Setting up the Gitlab Repository

First we need a Dockerfile from which your custom container is going to be build from and second a docker-compose.yml file which tells docker how to deploy the image (or images).

For this you can just clone your created git repository to your computer and add both files to your git root directory or just add them with commits through the Gitlab web interface.

Add .gitlab-ci.yml file to your repository

After that the only thing left to do is to setup a .gitlab-ci.yml file. This file is basically telling Gitlab what commands to run under which conditions in your repository. So we need to tell Gitlab that with every commit to master its supposed to build the docker container from the Dockerfile, push it to your docker repository and then deploy it to your server with a docker-compose.yml file through ssh.

Just copy this .gitlab-ci.yml file to your repository's root, if you want to build and Publish a container from Dockerfile and then deploy that with a docker-compose.yml file:

paulgoio/templates
gitlab mirror: templates to build and deploy docker images with docker-compose - paulgoio/templates
al..yml for deployment and building

OR

Copy this .gitlab-ci.yml file if you just want to build and publish a container with a Dockerfile:

paulgoio/templates
gitlab mirror: templates to build and deploy docker images with docker-compose - paulgoio/templates
build.yml just for building and publishing an image

OR

Copy this .gitlab-ci.yml file if you just want do deploy a docker-compose.yml file:

paulgoio/templates
gitlab mirror: templates to build and deploy docker images with docker-compose - paulgoio/templates
deploy.yml just for deploying an docker-compose.yml file to a server

Settings Repository Variables

So now you have all the files needed for your deployment, but one key thing is still missing: The variables telling your Gitlab CI/CD Pipeline where to deploy your files to and all the passwords for your server and Docker repository.

For that set the following variables for building and publishing images:

  • DOCKER_PASSWD : password for the Docker repository; For Dockerhub this would be an API key.
  • DOCKER_USER : user used to authenticate with repository.
  • DOCKER_REPO : repository name that the image gets pushed to (For Dockerhub this would be the account name; test would push to test/repo-name:latest)

Set the following variables for deployment through ssh on your server:

  • SSH_HOST : the IP or a DNS name pointing to the deployment server with ssh and docker as well as docker-compose installed
  • SSH_PORT : the ssh port on that deployment server
  • SSH_USER : ssh user used to deploy the services
  • SSH_PRIVATE_KEY : private key used to authenticate with deployment server; you can just generate a new key and copy the key with `ssh-copy-id user@server` to the server

The following variables you can optionally set in your docker-compose.yml file in the foll wing syntax: {VARIABLE_NAME} and also set them in your Gitlab repository normally:

  • DOMAIN : is meant for your main domain [www.example.org], is going to be set for the environment
  • STAGING_DOMAIN : is meant for your staging domain [www.staging.example.org], is going to be set for the environment
  • KEY1 KEY2 KEY3 : any password or key that can be set in your docker-compose file
  • EMAIL : reserved for an Email

Running your first Pipeline

Now that you set your variables and added your Dockerfile, docker-compose.yml file and .gitlab-ci.yml file to your Gitlab repository you can run your first pipeline. For that just head to the CI/CD tab in your repository in the Gitlab web interface and from there open the Pipelines page. There you can start a manual pipeline with the "Run Pipeline" button. If everything is setup correctly you should see a deployment running which pushes a container to your repository and sets up the docker-compose.yml file on your server. Congrats!

Troubleshooting

If the pipeline didn't work, check that:

  • you set your variables for ssh and the docker repository
  • your docker-compose.yml and Dockerfile are valid (docker-compose.yaml does NOT work)
  • you have availible runners for your project (check under settings -> CI/CD -> Runners)
  • your .gitlab-ci.yml file is valid (you can check at CI/CD -> Editor is syntax is write)

Final Thoughts

Now you can easily push code to your repository and Gitlab is going to do all the heavy lifting for you. With this everything is managed in one place and you can even scheduled pipelines which run every week for example. Feel free to open an issue in the GitHub repository if something does not work!