Secrets are one of the sneakiest vulnerability issues you can have in a Docker image if you don't know how to handle them.
If you need to clone a private repository or to download a private package you must use sensitive data during your docker build, there's no easy way around that.
In this tutorial on the advanced usage of Docker series, I'll explain how to use a build secret in a safe way.
What is Buildkit
I explained last week what is the Buildkit build engine, how to set it up, and how you can use Buildkit to speed up docker build.
Never use COPY and rm
If you are dealing with secrets during your development, I'm sure the first thing you've tried is to COPY a file with credentials from your Dockerfile and then remove it with rm when you don't need it anymore...
This is so wrong
because you are just deleting the file from that layer but the credentials are still in the layer above!
Unsafe code example
Let's use this Dockerfile.
FROM ubuntu:bionic
COPY .netrc /
RUN rm .netrc
And let's build it.
$ docker build -t unsafe . -f Dockerfile.not-safe
Sending build context to Docker daemon 4.096kB
Step 1/3 : FROM ubuntu:bionic
---> c14bccfdea1c
Step 2/3 : COPY .netrc /
---> 18d1eb74c6da
Step 3/3 : RUN rm .netrc
---> Running in fafd31acf728
Removing intermediate container fafd31acf728
---> d7d4315738a6
Successfully built d7d4315738a6
Successfully tagged unsafe:latest
Now we want to use the command docker history to list all the layers of the image.
$ docker history d7d4315738a6
IMAGE CREATED CREATED BY SIZE COMMENT
d7d4315738a6 10 seconds ago /bin/sh -c rm .netrc 0B
18d1eb74c6da 14 seconds ago /bin/sh -c #(nop) COPY file:a0fa732884be950b… 19B
c14bccfdea1c 4 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 4 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do… 7B
<missing> 4 weeks ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 0B
<missing> 4 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /… 745B
<missing> 4 weeks ago /bin/sh -c #(nop) ADD file:84f8ddc4d76e1e867… 63.2MB
Here we can see the latest layer created, d7d4315738a6, but you don't care about it.
The best part is the previous layer, 18d1eb74c6da, which we can analyze deeper.
$ docker run -it 18d1eb74c6da
root@09fb719ec3dc:/# cat .netrc
my secret password
This is the deal: every layer of your image is available, including the ones with your secrets!
Think about it next time you do COPY and rm.
Buildkit to the rescue with --secret
Buildkit adds a new flag called --secret for the docker build command. You can use it to provide safely a secret to your Dockerfile at build time! Buildkit mounts the secret using tmpfs in a temporary file located in /run/secrets that we can use to access a secret in the Dockerfile.
Using this feature we are sure that no secrets will remain in the image!
Safe code example
Dockerfile
Let's use the following Dockerfile
# syntax = docker/dockerfile:1.0-experimental
FROM ubuntu:bionic
RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc
RUN cat /.netrc
The first thing to notice is # syntax = docker/dockerfile:1.0-experimental, we tell Docker to use the new syntax to exploit the new Buildkit functionality.
Then, with the first RUN command, the magic happens. We tell Docker to mount a secret with the id mynetrc to the destination /.netrc and in the same line, we execute the cat command just for the sake of the example.
Then we RUN again the cat command on the same file.
Build command
To build our Dockerfile this is the command:
$ DOCKER_BUILDKIT=1 docker build --secret id=mynetrc,src=.netrc --progress=plain --no-cache -f Dockerfile.safe -t safe .
You can notice here the flag --secret which tells Docker the secret name and location. We also need to set DOCKER_BUILDKIT=1 to use the Buildkit build engine.
OK, let's build it.
$ DOCKER_BUILDKIT=1 docker build --secret id=mynetrc,src=.netrc --progress=plain --no-cache -f Dockerfile.safe -t safe .
#...
#8 [1/3] FROM docker.io/library/ubuntu:bionic
#8 CACHED
#9 [2/3] RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc
#9 0.808 my secret password
#9 DONE 1.7s
#10 [3/3] RUN cat /.netrc
#10 DONE 2.0s
#11 exporting to image
#11 exporting layers
#11 exporting layers 0.7s done
#11 writing image sha256:b86ed6e0585c2f2e5cb14796b896dae0004f75004ccece0949a3de0ca600b113 0.0s done
#11 naming to docker.io/library/safe 0.0s done
#11 DONE 1.0s
As you can see, in the RUN --mount=type=secret,id=mynetrc,dst=/.netrc cat /.netrc we can access the content of the file, instead on the following RUN there's no output.
The .netrc file, in fact, is present in the final layer of the image but it's empty.
Let's use the command docker history to list all the layers of this new image.
$ docker history b86ed6e0585c
IMAGE CREATED CREATED BY SIZE COMMENT
b86ed6e0585c 5 hours ago RUN /bin/sh -c cat /.netrc # buildkit 0B buildkit.dockerfile.v0
<missing> 5 hours ago RUN /bin/sh -c cat /.netrc # buildkit 0B buildkit.dockerfile.v0
<missing> 4 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 4 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do… 7B
<missing> 4 weeks ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 0B
<missing> 4 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /… 745B
<missing> 4 weeks ago /bin/sh -c #(nop) ADD file:84f8ddc4d76e1e867… 63.2MB
As you can see, it's not possible now to load an older layer to read the secret.
This is it!
I hope this was useful for you, now go and refactor your old Dockerfile!
Reach me on Twitter @gasparevitta and let me know how you use manage secrets.
You can find the code snippets on Github.
This article was originally published on my blog. Head over there if you like this post and want to read others like it!
Top comments (5)
This is still experimental and shouldn't be used for production builds. If you want to use secrets you should use multi-stage builds. For example:
Yep, that's a very good method if you are using multistage build. I talk about it here gasparevitta.com/posts/advanced-do...
Very interesting and very useful.
Thank you for that article. 👍🏼
why not mount using -v??
You can use -v with docker run but not with docker build. Here I explain how to do when you need secrets at build time.