Docker setup for Go APIs

In this blog post I've covered how to integrate Docker in your Backend Go APIs application.

What you will gain?

After reading this, you will be able to create your own DockerFile and docker-compose file and have a up and running application.

Code for this post can be found on my github
Code: https://github.com/the-arcade-01/go-api

PreRequisites

You've gone through the 1st and 2nd blog of this series as we will be using the same MySQL api project.
You should have Docker installed in your system.

Concepts

Previously, we have install MySQL on our machine and we were interacting with it on our machine, but now we'll leverage docker and run our MySQL database and our Go application in a container.

First lets create our docker config for our Go API app

DockerFile contains steps which creates the image, which you can think of like a snapshot of your code, which you run on a container, Container means an isolated environment in which your application runs.

DockerFile

# STEP 1
FROM golang:1.21.3

# STEP 2
WORKDIR /app

# STEP 3
COPY go.mod go.sum ./

# STEP 4
RUN go mod download

# STEP 5
COPY . .

# STEP 6
RUN go build -o ./bin/main main.go

# STEP 7
RUN chmod +x ./run.sh

# STEP 8
ENTRYPOINT [ "sh", "./run.sh" ]

Now, lets describe each steps of above file

1. FROM: in this step we download the base version of OS or dependency on which our application will run. Here, we are downloading golang:1.21.3 from DockerHub
2. WORKDIR: here we are specifying in which folder our application will be present on this above base machine.
3. COPY <src> <dest>: Now, we are copying our go.mod and go.sum file which contains all the dependencies.
4. RUN <cmd>: now, we are download all the dependencies which are required for the Go app.
5. In this step, we are copying all the files into our docker base machine.
6. Now, we will generate the binary executable for our Go app.
7. Now, this we're giving executable permission to our bash file run.sh which looks like this

run.sh

#!/bin/sh

/app/bin/main

this bash file contains the path of our binary executable, and when we will run our bash file, our binary executable will be called.
ENTRYPOINT: this tells which command to execute when we start our container. Here, run.sh will be called.

Now, we can setup docker-compose file

docker-compose is a single file which contains all the instructions of different services which are being used in the current application. This file helps us to run all the services in a single command.

docker-compose.yaml

version: "3.8"

services:
  db:
    container_name: "mysql_db"
    image: mysql:8.0.33
    networks:
      - default
    restart: always
    ports:
      - "3306:3306"
    environment:
      MYSQL_RANDOM_ROOT_PASSWORD: ${MYSQL_RANDOM_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - ./db.sql:/docker-entrypoint-initdb.d/0_init.sql
      - mysql_data:/var/lib/mysql
    command: --default-authentication-plugin=mysql_native_password
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      timeout: 20s
      retries: 10
  app:
    container_name: "go_api"
    build: .
    env_file:
      - .env
    ports:
      - "5000:5000"
    depends_on:
      db:
        condition: service_healthy

networks:
  default:

volumes:
  mysql_data:

Lets go through each keyword
services: inside this we list all our services which we are using, like database, redis, go app etc. Inside this you will see two config, first db config and second app config.

db config contains everything related to our MySQL database.

• container_name: we assign a name to our container
• image: here we specify which base version we want to use, here we using mysql:8.0.33 version.
• networks: networks are use for establishing connection between multiple containers, here we want our Go app and db to connect to a same network, and are using default for that which will be created by docker itself when we run our file.
• restart: if we encounter any error will starting the container, it will automatically restart untils its up.
• ports: default mysql port 3306

NOTE: If you've MySQL currently running on your system, then you have to stop it or otherwise you will have connection error

Error starting userland proxy: listen tcp4 0.0.0.0:3306: bind: address already in use

On linux you can run the below command

sudo service mysql stop 

• environment: here we specify all the necessary env. variables as per MySQL docker docs
• volumes: volumes are used for presisting data like eg. you ran a MySQL container and wants its data to be save even when you kill that container. Here we are specify two things
• the below cmd will copy our db.sql file which contains our database and table schema queries, we are copying sql file in docker-entrypoint file. When we will run our MySQL container it will automatically run our db migrations.

- ./db.sql:/docker-entrypoint-initdb.d/0_init.sql

• path at which our volume will be present

- mysql_data:/var/lib/mysql

• command: specifying auth policy as mysql_native_password, as per docs its mentioned to use caching_sha2_password instead of this for version greater than 8.0, but this policy will also work fine.
• healthcheck: this is an important feature, now we want our Go api app to start after the db has been initialized and is running, we specify a cmd which will get trigger and based on its success our apps which are depended on db will start. Here, the cmd we are using is a simple ping.

Now, lets see our app config
Here we specify the container name go_api, docker build path to be . which will pick the DockerFile, then we specify the env_file which looks like this
.env

PORT=:5000
DATABASE_URL=<user>:<password>@tcp(<mysql-container>:3306)/<database>?parseTime=true
DB_DRIVER=mysql
MYSQL_RANDOM_ROOT_PASSWORD=<password>
MYSQL_DATABASE=<database>
MYSQL_USER=<user>
MYSQL_PASSWORD=<password>

NOTE: In the DATABASE_URL, <mysql-container> value should be replaced with db, eg: DATABASE_URL=user:123@tcp(db:3306)/Practice?parseTime=true

depends_on: this tells on which other containers our current app is depending on, and if the other container is not initialized or is inactive, then our container will not start. Here we have specified

depends_on:
      db:
        condition: service_healthy

Running the Application

Now, we are ready to run our application
So, first we'll run

docker compose build

this will build an image of our application.

➜  go-api git:(main) docker compose build
[+] Building 35.2s (12/12) FINISHED                              docker:default
 => [app internal] load .dockerignore                                      0.1s
 => => transferring context: 2B                                            0.0s
 => [app internal] load build definition from Dockerfile                   0.2s
 => => transferring dockerfile: 218B                                       0.0s
 => [app internal] load metadata for docker.io/library/golang:1.21.3      10.3s
 => [app internal] load build context                                      0.4s
 => => transferring context: 13.87kB                                       0.3s
 => [app 1/7] FROM docker.io/library/golang:1.21.3@sha256:b113af1e8b06f06  0.0s
 => CACHED [app 2/7] WORKDIR /app                                          0.0s
 => [app 3/7] COPY go.mod go.sum ./                                        0.5s
 => [app 4/7] RUN go mod download                                          3.5s
 => [app 5/7] COPY . .                                                     0.9s
 => [app 6/7] RUN go build -o ./bin/main main.go                          17.1s
 => [app 7/7] RUN chmod +x ./run.sh                                        1.1s
 => [app] exporting to image                                               0.7s
 => => exporting layers                                                    0.7s
 => => writing image sha256:bf43ed8cbf2418a6f75f9b4395a630557a2c8ee85db65  0.0s
 => => naming to docker.io/library/go-api-app                              0.1s 

Now, after the image has been build, we can run the below cmd to start our application

docker compose up

this will start our application and our MySQL container

➜  go-api git:(main) docker compose up              
[+] Running 2/2
 ✔ Container mysql_db  Created                                             0.0s 
 ✔ Container go_api    Recreated                                           0.4s 
Attaching to go_api, mysql_db

and

mysql_db  | 2023-12-21T10:35:29.836732Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.33'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
go_api    | Connected to DB
go_api    | server running on port:5000

Testing

Now, we can test the todo apis

➜  go-api git:(main) curl -X POST 'http://localhost:5000/todos' -d '{"task": "Learn Docker with Go", "completed": false}'
Todo added!!% 

fetch todos

➜  go-api git:(main) curl -X GET 'http://localhost:5000/todos' 
[{"id":1,"task":"Learn Docker with Go","completed":false,"created_at":"2023-12-21T10:52:41Z","updated_at":"2023-12-21T10:52:41Z"}]

Logs will look like this

go_api    | Connected to DB
go_api    | server running on port:5000
go_api    | 2023/12/21 10:52:22 "GET http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:41316 - 200 119B in 18.831971ms
go_api    | 2023/12/21 10:52:41 "POST http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:45632 - 200 12B in 29.13446ms
go_api    | 2023/12/21 10:53:10 "GET http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:57434 - 200 248B in 643.34µs

Extras

You can even check which all containers are running, using below cmd

➜  go-api git:(main) docker ps        
CONTAINER ID   IMAGE              COMMAND                  CREATED         STATUS                   PORTS                                                  NAMES
a8a9f20533cd   go-api-app         "sh ./run.sh"            2 minutes ago   Up 2 minutes             0.0.0.0:5000->5000/tcp, :::5000->5000/tcp              go_api
f6d7a0153ac0   mysql:8.0.33       "docker-entrypoint.s…"   3 days ago      Up 2 minutes (healthy)   0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp   mysql_db

and you can also ssh into our MySQL container using its container id

➜  go-api git:(main) docker exec -it f6d7a0153ac0 sh            
sh-4.4# 

Now, we will check in our mysql table whether the data is present or not. Use the same user and password which you defined in .env file.

sh-4.4# mysql -u user -p 
Enter password: 

Now, check data in table Todos

mysql> use Practice;
Database changed
mysql> show tables;
+--------------------+
| Tables_in_Practice |
+--------------------+
| Todos              |
+--------------------+
1 row in set (0.01 sec)
mysql> select * from Todos;
+----+----------------------+-----------+---------------------+---------------------+
| id | task                 | completed | created_at          | updated_at          |
+----+----------------------+-----------+---------------------+---------------------+
|  1 | Learn Docker with Go |         0 | 2023-12-21 10:52:41 | 2023-12-21 10:52:41 |
+----+----------------------+-----------+---------------------+---------------------+
1 rows in set (0.00 sec)

Conclusion

That's it, we covered a lot of stuff in this blog, I encourage you to read more about Docker and create your own personal project using it.

Github: https://github.com/the-arcade-01/go-api

Thanks for reading till the end, really appreciate it!!

Comments

[Wolfgang Ellsässer]

While this setup could be useful for a first dev/test environment, please don't use this ever in prod or on a public server. While everything in your example is correct, it may be dangerous for inexperienced users. Depending on the setup, there may be some severe security issues:

The DB port is exposed directly to the internet. In most cases, you don't need this. You can easily fix that by removing

    ports:
      - "3306:3306"

from the Compose file.

You don't need the whole Go dev environment in your container. The best option would be to make a multi stage Dockerfiles and use a SCRATCH, Distroless, or Chainguard image for the productin build.

The API service runs a root, which ist just not neccesary. You could define an unprivileged user in the Dockerfile or in the Compose file.

You don't need a shell script execute the Go binary. Just run the binary directly with ENTRYPOINT.

[Aashish Koshti]

Completely agree with your points, i wrote this considering local/dev env., in prod mulitple things have to be change like correct db privlieges to specific app, multistage build for diff. env etc.
And yes, that running binary from ENTRYPOINT can be done, i actually was performing some steps before executing the binary so i used shell script.