Using Docker Compose
Overview
Docker Compose deploys and manages multi-container applications (microservices) on Docker nodes running in single-engine mode.
In contrast, Docker Swarm deploys and manages multi-container apps on Docker nodes running in swarm mode.
Docker Compose allows you to describe an entire app in a single declarative configuration file and deploy it with a single command. In other words, include all your run commands in one file, rather than having to enter them each time you run a container.
This is much more convenient than setting up VMs with Vagrant. For example, to test an app, you’d just need to:
- clone a git repo (containing a Dockerfile and a docker-compose.yml)
- run
docker-compose up
Then you have access to everything without having to install and configure databases.
Once the app is deployed, you can manage its entire lifecycle with a simple set of commands. And the configuration file can be stored in a VCS.
To check Docker Compose is installed, run: docker-compose --version
. It should be installed by default with Docker Desktop on Mac and Windows.
Creating Docker Compose Files
Docker Compose uses YAML files to define multi-service applications. You can also use JSON.
The default name for a Docker Compose YAML file is docker-compose.yml
. You can also use the -f
flag to specify custom filenames.
Here’s an example of a Docker Compose file:
version: "3.8"
services:
web-fe:
build: .
command: python app.py
ports:
- target: 5000
published: 5000
networks:
- counter-net
volumes:
- type: volume
source: counter-vol
target: /code
redis:
image: "redis:alpine"
networks:
counter-net
networks:
counter-net:
volumes:
counter-vol:
The version
key is no longer mandatory, but you’ll still see it around. This defines the version of the Compose file format (i.e. the API), not the version of Docker Compose or Docker Engine.
The services
key is where you define the different application microservices. In this case, there are two microservices: web-fe
and redis
.
You can include a depends_on
key to specify dependencies. For example:
depends_on:
- mysql-primary
- mysql-secondary
And you can add environment variables, such as passwords:
environment:
- MYSQL_PASSWORD=password
- POSTGRES_PASSWORD=password
These can also be specified in dictionary format. For example:
environment:
MYSQL_PASSWORD: password
POSTGRES_PASSWORD: password
The top-level networks
key tells Docker to create new networks. By default, Compose creates bridge networks. These are single-host networks that can connect only containers on the same Docker host. You can use the driver property to specify different network types. For example:
networks:
over-net:
driver: overlay
attachable: true
The top-level volumes
key is where you tell Docker to create new volumes.
In the example above, the services
section has two second-level keys:
- web-fe
- redis
Each of these defines a service (container) in the app. Within the definition of the web-fe
service, you give Docker the following instructions:
build: .
- this tells Docker to build a new image using the instructions in the Dockerfile in the current directory (.). The newly built image is used in a later stage to create the container for this service.command: python app.py
- this tells Docker to run a Python app calledapp.py
as the main app in the container. This can also be specified in the Dockerfile, although you can override it here in the Compose file. Theapp.py
file must exist in the image, and the image must contain Python. This is handled by the Dockerfile.ports:
- tells Docker to map port 5000 inside the container (-target
) to port 5000 on the host (published
). Traffic sent to the Docker host on port 5000 is directed to port 5000 on the container. The app inside the container listens on port 5000.networks:
- tells Docker which network to attach the service’s container to. The network should already exist or be defined in thenetworks
top-level key. If it’s an overlay network, it needs to have theattachable
flag.volumes:
- tells Docker to mount thecounter-vol
volume (source:) to /code (target:) inside the container. Thecounter-vol
needs to already exist, or be define in thevolumes
top-level key at the bottom of the file.
As both services are deployed to the same counter-net
network, they will be able to resolve each other by name.
Deploying an App with Compose
Compose uses the name of your directory as the project name. If your directory is called counter-app
, all resources names are prepended with counter-app_
.
You can override the default name Compose uses. That’s how you can run many copies of the same application in different sets of containers on a single Docker Engine.
The command to deploy an app is:
docker-compose up
You can add the -d
flag to run the app in the background.
Once the app is built and running, you can use normal docker
commands to view the images, containers, networks, and volumes created by Compose. For example:
- docker volume ls
- docker network ls
Managing an App with Compose
To see the current state of the app, use docker-compose ps
.
Use docker-compose top
to list the processes running inside of each service (container).
To stop a Compose app, use docker-compose stop
. This command just stops the app’s containers. The application definition remains on the system.
You can restart the app with the docker-compose restart
command.
You can delete a stopped Compose app with docker-compose rm
. This deletes the containers and networks the app is using, but doesn’t delete volumes or images.
Use the docker-compose down
command to stop and delete the app with a single command. Then only the images, volumes, and source code remain. As the images are still present, future deployments will be faster.
If the app’s code resides on a Docker volume, this means we can make changes to file in the volume from outside the container. Those changes are then reflected immediately in the app.
To remove volumes when you stop the app, use the following command:
docker-compose down -v
Creating Health and Dependency Checks in Docker Compose
As with Dockerfiles, you can add healthchecks to Docker Compose files:
healthcheck:
interval: 5s
timeout: 1s
retries: 2
start_period: 5s
interval
- time between checkstimeout
- how long the check should be allowed to run before it’s considered a failureretries
- number of consecutive failures allowed before the container is flagged as unhealthystart_period
- amount of time to wait before triggering the health check - gives your app some startup time before health checks run.
You can also add a health check in your Compose file for containers that don’t have one declared in the image.
Building an Image with Docker Compose
If you include a build
key, in your docker-compose.yml file, you can build an image before running it:
version: '2'
services:
frontend:
image: my-fe
build: .
In the example above, Docker checks for the image in cache. If it’s not there, Docker looks for a Dockerfile in the current directory and builds the image.
To ensure any existing images are rebuilt, use:
docker-compose up --build
(you can also specify a service name )
If you don’t specify a name for your image, Docker assigns one based on your project name. In this case, you can also remove the image with the following command:
docker-compose down --rmi local
Overriding docker-compose Files
If you want to get fancy, you can use a basic docker-compose.yml
file and then override it with values in an additional file called docker-compose.override.yml
.
Here’s the basic docker-compose.yml
:
version: '3.1'
services:
drupal:
image: drupal
postgres:
image: postgres
You can then include additional configuration with a docker-compose.override.yml
file:
version: '3.1'
services:
drupal:
ports:
- "8080:80"
volumes:
- drupal-modules:/var/www/html/modules
postgres:
environment:
- POSTGRES_PASSWORD=password
There are no images specified in the second file - that’s because they’re already in the main docker-compose.yml
file.
You could use a similar system for using different values for testing and production environments. For example, a docker-compose.test.yml
file would use test data, rather than mounting a real volume. This would make it faster. The docker-compose.prod.yml
then attaches the real volumes and live databases.
To run these custom files on top of the basic file, use the following format:
docker-compose -f docker-compose.yml docker-compose.test.yml -d
You can also view the consolidated file before you deploy it:
docker-compose -f docker-compose.yml docker-compose.test.yml config
To actually consolidate those two files into a new file:
docker-compose -f docker-compose.yml docker-compose.test.yml > output.yml
Limitations of Docker Compose
You get the desired state of your application when you run docker-compose up
, but that’s where Docker Compose ends. It’s not a full container platform like Docker Swarm or Kubernetes - it doesn’t continually run to ensure your application is maintained in its desired state.
If containers fail or you remove them manually, Docker Compose won’t restart or replace them until you explicitly run docker-compose up
again.
Docker Compose is great for running containers on a single machine, but that doesn’t work in a production environment. If that machine goes offline, you lose all your applications.
This is why you need an orchestrator, such as Docker Swarm or Kubernetes.