Docker multi-stage build and environment variables

This post is just a small compendium of links and ideas I found somewhere else while I was working on the following problem:

How to pass variables from a docker-compose file into a build stage in the Dockerfile?

The context

It turns out environment variables can be tricky, there are a few different levels and abstractions for each one of them. For example, if docker-compose.yaml builds a service from a Dockerfile stage and inside the container we have our own javascript framework with a .env or some defaults, we have the following levels of environment variables:

1. docker-compose.yaml
2. Dockerfile
3. Container environment files
4. NPM script environment variables
5. Application defaults

If you are like me, it can get messy after a while.

The flow

First let's get the javascript part out of the way:

1. Already existing vars have precedence over .env vars unless you programmatically decide not to.
2. NPM script vars take precedence over container environment vars.

Now, the Docker part. According to the docker-compose file documentation, available here:

1. env_file add environment variables from a file.
2. environment add environment variables... but, in the note at the end of the section it mentions that:

If your service specifies a build option, variables defined in environment are not automatically visible during the build. Use the args sub-option of build to define build-time environment variables.

1. args add build arguments, which are environment variables accessible only during the build process.

Well, it is very clear, once you read it. But it is common to assume how things work or to ignore that this document is out there, as you can see from issue #1837.

At this point we know how to pass a variable from the docker-compose.yaml to the Dockerfile, but that does not mean that this ARG will be available inside of the container. To achieve that we need to assign the ARG to ENV as follows:

ARG CONT_IMG_VER
ENV CONT_IMG_VER $CONT_IMG_VER

This time the source is the Dockerfile reference. There is a catch that is discussed after that section, you have to be aware of the cache miss on the first use of the ARG.

As you can see, it is not that hard to get it right, but you need to be aware of a few conventions that are not as intuitive as it might seem. Hope this save you a few minutes.

Finally, here is an interesting post I found while looking for this, Docker ARG, ENV and .env - a Complete Guide.

Cover image by Yohann Mourre on Unsplash

Comments

[Frank Font]

There are several twists on passing values into build time and runtime docker processes. This article is great for anyone tinkering with what's available.

My friends and I did some hybrid things to organize the potential docker chaos by creating this framework... github.com/bah-insignia/zcmd