Originally posted on Hint's blog.
As a software consultancy, we switch between many projects throughout the year. A critical factor in delivering value is the ease at which we are able to move between projects.
Over the years, we have used many tools to manage dependencies needed to run and develop our clients' projects. The problem with most tools has been the ability to have consistent, reproducible development environments across our team. About two years ago, we discovered that Docker was a viable option for building consistent development environments. Since then, we continue to iterate on our configuration as we learn new ways to handle the complexity of the projects while simplifying the setup process for our team.
In this guide, we will cover the basics of our Docker development environment for Rails.
Getting started
If you would like to follow along, install Docker CE and create, clone, or have a working Rails app.
We will be using a combination of Dockerfiles, Docker Compose, and bash scripts throughout this guide, so let's make a place for most of those files to live.
Start by creating a docker folder in the root of the Rails project. Here we will store Dockerfiles and bash scripts to be referenced from the docker-compose.yml file we will be creating later.
Inside the docker folder, create another folder named ruby. Inside the newly created ruby folder, create a file named Dockerfile. This file will contain commands to build a custom image for your Rails app.
Hello Dockerfile
ARG RUBY_VERSION=2.6
FROM ruby:$RUBY_VERSION
ARG DEBIAN_FRONTEND=noninteractive
In this block, we set the RUBY_VERSION and DEBIAN_FRONTEND build arguments and specify the docker image we will use as our base image.
The first ARG sets a default value for RUBY_VERSION, but passing in a value from the command line or a docker-compose file will override it, as you'll see later in the guide. An ARG defined before FROM is only available for use in FROM.
FROM in a Dockerfile is the base image for building our image and the start of the build stage. In our case, we are using the official Ruby image from Docker Hub, defaulting to Ruby 2.6.
The next ARG is used to set the build stage shell to noninteractive mode, which is the general expectation during the build process. We don't want this environment variable to carry over to when we are using the images in our development environment, which is why we are using ARG instead of ENV.
Base Software Install
ARG NODE_VERSION=11
RUN curl -sL https://deb.nodesource.com/setup_$NODE_VERSION.x | bash -
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y \
build-essential \
nodejs \
yarn \
locales \
git \
netcat \
vim \
sudo
Here we are setting up the necessary software and tools for running a modern Rails app. Like in the previous section, we are setting a default for our NODE_VERSION build argument.
The next two RUN lines set up the defined version of the node apt repository and the latest stable yarn apt repo. Since Webpacker started being officially supported in Rails 5.1, it is important we have a recent version of node and yarn available in the image we will be running Rails on.
The third RUN will look familiar if you have used any Debian based OS. It updates the apt repositories, which is important since we just installed two new ones, and then it installs our base software and tools.
I want to point out netcat and sudo specifically. netcat is a networking tool we will use to verify the other services are up when we are bringing up our Rails app through Docker Compose. We install sudo since by default it is not installed on the Debian based Docker images, and we will be using a non-root user in our Docker image.
Non-root User
ARG UID
ENV UID $UID
ARG GID
ENV GID $GID
ARG USER=ruby
ENV USER $USER
RUN groupadd -g $GID $USER && \
useradd -u $UID -g $USER -m $USER && \
usermod -p "*" $USER && \
usermod -aG sudo $USER && \
echo "$USER ALL=NOPASSWD: ALL" >> /etc/sudoers.d/50-$USER
Docker containers generally use the root user, which is not inherently bad, but is problematic for file permissions in a development environment. Our solution for this is to create a non-root user and pass in our UID, GID, and username as build arguments.
If we pass in our UID and GID, all files created or modified by the user in the container will share the same permissions as our user on the host machine.
You will notice here that we use the build arguments to set the environment variable (via ENV) since we will also want these variables available when we bring up the container.
In the RUN instruction we add a standard Linux user/group and then we add the new user to the sudoers file with no password. This gives us all the benefits of running as root while keeping file permissions correct.
Ruby, RubyGems, and Bundler Defaults
ENV LANG C.UTF-8
ENV BUNDLE_PATH /gems
ENV BUNDLE_HOME /gems
ARG BUNDLE_JOBS=20
ENV BUNDLE_JOBS $BUNDLE_JOBS
ARG BUNDLE_RETRY=5
ENV BUNDLE_RETRY $BUNDLE_RETRY
ENV GEM_HOME /gems
ENV GEM_PATH /gems
ENV PATH /gems/bin:$PATH
Explicitly setting the LANG environment variable specifies the fallback locale setting for the image. UTF-8 is a sane fallback and is what locale defaults to when it's working properly.
We will be using a volume with Compose, so we need to point RubyGems and Bundler to where that volume will mount in the file system. We also set the gem executables in the path and set some defaults for Bundler, which are configurable via build arguments.
Optional Software Install
#-----------------
# Postgres Client:
#-----------------
ARG INSTALL_PG_CLIENT=false
RUN if [ "$INSTALL_PG_CLIENT" = true ]; then \
apt-get install -y postgresql-client \
;fi
Here is an example of how to set up optional software installs in the Dockerfile. In this scenario, postgresql-client will not be installed by default, but will be installed if we pass a build argument set to true (INSTALL_PG_CLIENT=true).
Dockerfile Final Touches
RUN mkdir -p "$GEM_HOME" && chown $USER:$USER "$GEM_HOME"
RUN mkdir -p /app && chown $USER:$USER /app
WORKDIR /app
RUN mkdir -p node_modules && chown $USER:$USER node_modules
RUN mkdir -p public/packs && chown $USER:$USER public/packs
RUN mkdir -p tmp/cache && chown $USER:$USER tmp/cache
USER $USER
RUN gem install bundler
To wrap up the Dockerfile, we create and set permissions on needed directories that were referenced previously in the file or that we will be setting up as volumes with Docker Compose.
We call USER $USER here at the end of the file, which will set the user for the image when you boot it as a container.
The last RUN command installs Bundler, which may not be required depending on the version of Ruby.
Next, we'll take a look at our docker-compose.yml file.
Docker Compose
Docker's definition of Compose is a great place to start.
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration.
That is what we are going to be doing in this section of the guide, adding support for a multi-container Docker application.
The first thing we will need is a docker-compose.yml file at the root of the Rails app. If you would like to copy and paste the following YAML into a docker-compose.yml file, I'll breakdown its parts below.
version: '3.7'
services:
rails:
build:
context: ./docker/ruby
args:
- RUBY_VERSION=2.6
- BUNDLE_JOBS=15
- BUNDLE_RETRY=2
- NODE_VERSION=12
- INSTALL_PG_CLIENT=true
- UID=500
- GID=500
environment:
- DATABASE_USER=postgres
- DATABASE_HOST=postgres
command: bundle exec rails server -p 3000 -b '0.0.0.0'
entrypoint: docker/ruby/entrypoint.sh
volumes:
- .:/app:cached
- gems:/gems
- node_modules:/app/node_modules
- packs:/app/public/packs
- rails_cache:/app/tmp/cache
ports:
- "3000:3000"
user: ruby
tty: true
stdin_open: true
depends_on:
- postgres
postgres:
image: postgres:11
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
volumes:
- postgres:/var/lib/postgresql/data
volumes:
gems:
postgres:
node_modules:
packs:
rails_cache:
In this file, we are defining two services: the rails service, which our Rails app will run in, and the postgres service, which will accommodate PostgreSQL. The names for these services are arbitrary and could easily be foo and bar, but since the service names are used for building, starting, stopping, and networking, I would recommend naming them close to the actual service they are running.
We also are setting our Compose file compatibility to version: '3.7' which is the latest at the time of this writing.
The App Service Broken Down
version: '3.7'
services:
rails:
build:
context: ./docker/ruby
args:
- RUBY_VERSION=2.6
- BUNDLE_JOBS=15
- BUNDLE_RETRY=2
- NODE_VERSION=12
- INSTALL_PG_CLIENT=true
- UID=500
- GID=500
The first part of the rails service is to set up to build the image from the Dockerfile we created earlier. Once built, it uses the local image from that point forward. We set the context, which is the directory path where our Dockerfile is stored. The args: key specifies the build arguments we set up in the Dockerfile earlier in the guide. Notice that we are overriding some of our earlier defaults here.
services:
rails:
. . .
environment:
- DATABASE_USER=postgres
- DATABASE_HOST=postgres
command: bundle exec rails server -p 3000 -b '0.0.0.0'
entrypoint: docker/ruby/entrypoint.sh
volumes:
- .:/app:cached
- gems:/gems
- node_modules:/app/node_modules
- packs:/app/public/packs
- rails_cache:/app/tmp/cache
ports:
- "3000:3000"
In the next part, we set up environment variables for connecting to the postgres service, which will require updating your database.ymlexample file to take advantage of these variables.
We also set the default command that will run when docker-compose up runs, which in this case starts the Rails server on port 3000 and binds it to all IP addresses.
Next, we point to our entrypoint script, which we'll revisit after breaking down the rest of the Compose file.
After the entrypoint, we set up volumes.
The first volume mounts the current directory (.) to /app in the container. The mapping for this one is HOST:CONTAINER. If you are on Mac, I would recommend using .:/app:cached since file sharing on Docker for Mac is CPU bound. By setting :cached on the mount point it allows files to be out of sync with the host being authoritative. Here are more details about file sharing performance on Docker for Mac. In our testing :cached has been the best option for speed vs. tradeoffs.
The next volumes are named volumes. Creating them happens when we bring up the containers for the first time and then they are persistent from up and down. These volumes are native to the Docker environment, so they operate at native speeds. The persistence and speed are why we have chosen to use them for gem, node_modules, packs, and rails_cache storage; otherwise, we would have to reinstall both every time we bring our environment back up. Lastly, there is a top-level volumes key at the very bottom of the docker-compose.yml file, which is where they are defined.
The last key in this section is ports. Ports only need to be defined to access the containers from the host otherwise container to container communication happens on the internal Docker network generally using service names.
The mapping "3000:3000" allows us to connect to the Rails server at [localhost:3000](http://localhost:3000). The mapping is HOST:CONTAINER just like the volume mount, and it is recommended to pass them in as strings because YAML parses numbers in xx:yy as base-60.
services:
rails:
. . .
user: ruby
tty: true
stdin_open: true
depends_on:
- postgres
This last section explicitly sets the user to ruby, which was set up in our Dockerfile above.
Use the next two keys: tty and stdin_open, for debugging with binding.pry while hitting the Rails server. Check out this gist for more info.
One thing to call out from that gist is to use ctrl-p + ctrl-q to detach from the Docker container and leave it in a running state. The depends_on key is used to declare other containers that are required to start with the service. Currently, it is only dependent on postgres. No other health checks or validations are run; we will handle those in the entrypoint.
The Postgres Service
services:
rails:
. . .
postgres:
image: postgres:11
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
volumes:
- postgres:/var/lib/postgresql/data
We define our postgres service next with pretty minimal configuration compared to what we went through for the rails service.
The first key, image, will check locally first, then download the official postgres image from Docker Hub with a matching tag if needed. In this case, it will pull in 11.5. I chose 11 for this example because that is now the default on Heroku, but there are lots of image options available on Docker Hub.
We are using a named volume here as well for our postgres data.
And, that wraps up the Compose file.
Enter The Entrypoint
There are a few requirements for starting a Rails server, e.g., the database running and accepting connections. We use the entrypoint, which is a bash script, to fulfill those requirements. Looking back at the compose file we should create this bash script at docker/ruby/entrypoint.sh and make it executable.
#! /bin/bash
set -e
: ${APP_PATH:="/app"}
: ${APP_TEMP_PATH:="$APP_PATH/tmp"}
: ${APP_SETUP_LOCK:="$APP_TEMP_PATH/setup.lock"}
: ${APP_SETUP_WAIT:="5"}
# 1: Define the functions to lock and unlock our app container's setup
# processes:
function lock_setup { mkdir -p $APP_TEMP_PATH && touch $APP_SETUP_LOCK; }
function unlock_setup { rm -rf $APP_SETUP_LOCK; }
function wait_setup { echo "Waiting for app setup to finish..."; sleep $APP_SETUP_WAIT; }
# 2: 'Unlock' the setup process if the script exits prematurely:
trap unlock_setup HUP INT QUIT KILL TERM EXIT
# 3: Wait for postgres to come up
echo "DB is not ready, sleeping..."
until nc -vz postgres 5432 &>/dev/null; do
sleep 1
done
echo "DB is ready, starting Rails."
# 4: Specify a default command, in case it wasn't issued:
if [ -z "$1" ]; then set -- bundle exec rails server -p 3000 -b 0.0.0.0 "$@"; fi
# 5: Run the checks only if the app code is executed:
if [[ "$3" = "rails" ]]
then
# Clean up any orphaned lock file
unlock_setup
# 6: Wait until the setup 'lock' file no longer exists:
while [ -f $APP_SETUP_LOCK ]; do wait_setup; done
# 7: 'Lock' the setup process, to prevent a race condition when the
# project's app containers will try to install gems and set up the
# database concurrently:
lock_setup
# 8: Check if dependencies need to be installed and install them
bundle install
yarn install
# 9: Run migrations or set up the database if it doesn't exist
# Rails >= 6
bundle exec rails db:prepare
# Rails < 6
# bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:setup
# 10: 'Unlock' the setup process:
unlock_setup
# 11: If the command to execute is 'rails server', then we must remove any
# pid file present. Suddenly killing and removing app containers might leave
# this file, and prevent rails from starting-up if present:
if [[ "$4" = "s" || "$4" = "server" ]]; then rm -rf /app/tmp/pids/server.pid; fi
fi
# 12: Replace the shell with the given command:
exec "$@"
I will only call out a few things specific to working with Rails from this file. Below, we are using nc(netcat) to verify that postgres is up before running any database commands. nc doesn't just ping the server; it is checking that the service is responding on a specific port. This check runs every second until it boots. Note: We are using the service name postgres to connect to the container.
echo "DB is not ready, sleeping..."
until nc -vz postgres 5432 &>/dev/null; do
sleep 1
done
echo "DB is ready, starting Rails."
Next, we run our bundle and yarn install commands to make sure we have the latest dependencies. Once both of those have run, we will use the new Rails 6 db:prepare method to either set up the database or run migrations. This can be handled in many different ways but my preference is to use Rails tools. (Note: the comment is how you would do it on Rails 5 or older)
bundle install
yarn install
# Rails >= 6
bundle exec rails db:prepare
# Rails < 6
# bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:setup
Lastly, we check for any dangling PID files left from killing and removing the app containers. If the PID files are left our rails server command will fail.
if [[ "$4" = "s" || "$4" = "server" ]]; then rm -rf /app/tmp/pids/server.pid; fi
Start It Up
With all of this in place, we run docker-compose up, which will build and start our containers. It is also a great time to get coffee because building the app container will take some time. Once everything is built and up, point your browser at localhost:3000 and your homepage, or if this is a new app, it will display the default Rails homepage. If you need to run specs or any other Rails command, running docker-compose exec app bash in a separate terminal will launch a Bash session on the running app container.
Wrapping Up
We now have a base-level Rails Docker development environment. Here is a gist of all the three files referenced in this guide. Now that you have an understanding of how to set up a Docker development environment, check out Railsdock! It is a CLI tool we are working on that will generate most of this config for you. PR's appreciated!
In the next part of the series, we will set up more services and use Compose features to share config between similar services.
Top comments (5)
If you're a fan of TL;DR's, as Nate linked to, check out Railsdock.
Nate and I have been hard at work demystifying all things Docker so feel free to post comments/questions/PR's and we'll do what we can to shed light on the subject.
A tool for generating Ruby on Rails development environment Docker configuration
A command line tool for Dockerizing your Rails development environment.
Prerequisite
The configuration generated by this gem depends on Docker Desktop. Make sure it is installed before continuing.
Installation
Install the gem:
Or add it to your Gemfile:
Then run:
Usage
Navigate to your project directory:
Then run:
The gem will then walk you through configuring Docker by asking questions about your environment.
Development
After checking out the repo, run
bin/setupto install dependencies. Then, runrake specto run the tests. You can also runbin/consolefor an interactive prompt that will allow you to experiment.To install this gem onto your local machine, run
bundle exec rake install. To release a new version, update the version number inversion.rb, and then runbundle…Thanks for the post. It is surprisingly hard to find real good posts on setting up Rails with Docker and this is a great post on the topic. Going to add couple of these little gems to my current Docker setup for my Docker setup.
Do you use the same Dockerfile for production?
We build a base image that we build dev and production from. I plan to do follow up posts. I’ll keep in mind doing one about production as well.
This is great! I've been tinkering with a docker setup for development for a while now. Even have a few that I have implemented. None of them are elegant as this. In particular, I've been struggling with some of the issues you tackled in your entrypoint.sh. This is a keeper.
Two notes:
1) the location of the three files (Dockerfile, docker-compose.yml, and entrypoint.sh) are inconsistent as specified in the blog post and referenced in the docker-compose.yml file.
2) a reference to the completed files (Dockerfile, docker-compose.yml, and entrypoint.sh) would be helpful so I'm including a gist to that effect: gist.github.com/joshillian/d601a2e...
Thanks, Josh!
I had those notes while on vacation and haven’t fixed the post. Thanks for the reminder. 😁