Docker in Docker access for non-root users

Mounting a host machine docker daemon within a docker container is really as simple as starting the container with a volume pointing at the hosts docker socket, for example:

docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock \
    ubuntu /bin/bash

The socket is mounted, huzzah! From here the user could install docker with apt install docker.io and be on their way. This method should work on both Linux and Mac machines. Windows machines a bit more complicated because the docker daemon needs to be remotely enabled to be accessed via a tcp connection. I'm not going to cover Windows here.

To add some complexity, running docker as a non-root user without doing a privilege escalation (like sudo) won't "just work" as the above example does. Beyond creating a dockerfile, mounting the docker socket, and creating a user there are bunch of permission "things" that need to be done to allow access.

The thing that may have brought you here is this error:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/json: dial unix /var/run/docker.sock: connect: permission denied

So, why wouldn't you want to run the docker container as root? Wellllll, there are a few reasons, security, sandboxing, not giving the user root is generally a good thing. Though there are other reasons. I went down this rabbit hole because I wanted to build and run some tests within the container, these tests verified some created files were read only, but as root, no matter the permission, they were always readable which failed the test. This is different then our production configuration which worked correctly, so finding a way to get the container to mimic a production deployment was essential for having confidence in the tests.

Okay, let's get into it. The docker file will look something like:

# dockerfile:
FROM ubuntu:jammy

# install the docker client and sudo
RUN apt update && apt install -y docker.io sudo

# add 'myuser' with bash as the default terminal
# and setup the user home tree
RUN useradd myuser -s /bin/bash -m

# copy in our entrypoint script
COPY docker-entrypoint.sh /docker-entrypoint.sh

# initiate our entry point script
ENTRYPOINT ["docker-entrypoint.sh"]

It's important to note that we don't set the current user before running the dockerfile. This is because we need to setup the aformentioned permission "things" as root then we can move to execute the workload as the myuser user.

The entrypoint script:

# docker-entrypoint.sh:
#!/usr/bin/env bash

# verify the docker.sock is present in the container
# if this fails it's because the socket wasn't mounted
if [[! -S /var/run/docker.sock]]; then
    echo "docker socket is missing"
    exit 1
fi

# verify the docker.sock has read and write permissions enabled
if [[! -r /var/run/docker.sock || ! -w /var/run/docker.sock]]
then
    echo "docker socket is missing read/write permission"
    exit 1
fi

# get the group id of the docker socket
# this is inherited from the host machine
gid=$(stat -c '%g' '/var/run/docker.sock')

# lookup the group by id within the container
# if it's missing, swallow the error and create the group
# using the inherited socket group id called 'docker'
if ! getent group "$gid" >/dev/null; then
    addgroup --gid "$gid" docker
fi

# get the name of the group by group id.
# this doesn't necessarily have to be called 'docker'
# it could have different names on both the host machine
# and container due to system differences or group id collisions
gname=$(getent group "$gid" | cut -d: -f1)

# check if the group name is in the list of groups that
# 'myuser' has membership too. If not, add them.
if ! groups myuser | grep --quiet "\b${gname}\b"; then
    usermod --append --groups "$gid" myuser
fi

# finally switch to our user and use the legacy buildkit.
# it's important to use the legacy buildkit because of version
# mismatching running on Docker for Desktop and a minimal linux
# installation.
sudo -u myuser DOCKER_BUILDKIT=0 "$@"

Most of the script seems pretty straight forward until we arrive at the permission "things". This section of the script is super important because non-root users are unlikely to be in the correct group by default to have execution read/write privilege to the docker socket which brings it's permissions from the host machine.

This seems pretty hacky (and it kinda is) however, the alternative which popped in my mind is why don't we just change the permissions on the mounted docker socket? Well, because that would remove the ability for the daemon user on the host machine to continue working on it and effectively break until those permission are reset.

Finally, building and running the container is pretty straight forward, building:

docker build -t docker-in-docker .

and running has two possible parameters:

# linux
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock \
    docker-in-docker some-command

# macos (note the '.raw' on the end of the docker.sock)
# this .raw isnt necessary if running as the root user
# but is if you're not
docker run -it --rm -v /var/run/docker.sock.raw:/var/run/docker.sock \
    docker-in-docker some-command

As a small disclaimer, the techniques I covered here are great for testing, CI/CD tooling, one-off's, but definitely should not be run as a standard production environment. Mounting docker in docker brings in all sorts of risk and issues.

Finally, I'd like to give a quick shout out to my colleague and friend Kamal who got me through the permission chunk of this!