Introduction
Hello, friends! 😉 Welcome to a really great tutorial. I've tried to make for you as simple step-by-step instructions as possible, based on a real-life application, so that you can apply this knowledge here and now.
I intentionally don't want to divide this tutorial into several disjointed parts, so that you don't lose the thought and focus. After all, I'm writing this tutorial only to share my experience and to show that backend development in Golang using the Fiber framework is easy!
At the end of the tutorial you will find a self-check block of knowledge, as well as a plan for further development of the application. So, I suggest you save the link to this tutorial to your bookmarks and share it on your social networks.
❤️ Like, 🦄 Unicorn, 🔖 Bookmark and let's go!
📝 Table of contents
- What do we want to build?
- My approach to Go project architecture
- Project configuration
- Practical part
- Testing the application
- Run project locally
- A self-check block of knowledge
- Plan for further development
What do we want to build?
Let's create a REST API for an online library application in which we create new books, view them, and update & delete their information. But some methods will require us to authorize through providing a valid JWT access token. I'll store all the information about the books, as usual, in my beloved PostgreSQL.
I think, this functionality is enough to help you understand, how easy it is to work with Fiber web framework to create a REST API in Go.
API methods
Public:
-
GET:
/api/v1/books
, get all books; -
GET:
/api/v1/book/{id}
, get book by given ID; -
GET:
/api/v1/token/new
, create a new access token (for a demo);
Private (JWT protected):
-
POST:
/api/v1/book
, create a new book; -
PATCH:
/api/v1/book
, update an existing book; -
DELETE:
/api/v1/book
, delete an existing book;
Full application code for advanced users
If you feel strong enough to figure out the code yourself, the entire draft of this application is published in my GitHub repository:
koddr / tutorial-go-fiber-rest-api
📖 Build a RESTful API on Go: Fiber, PostgreSQL, JWT and Swagger docs in isolated Docker containers.
📖 Tutorial: Build a RESTful API on Go
Fiber, PostgreSQL, JWT and Swagger docs in isolated Docker containers.
👉 The full article is published on March 22, 2021, on Dev.to: https://dev.to/koddr/build-a-restful-api-on-go-fiber-postgresql-jwt-and-swagger-docs-in-isolated-docker-containers-475j
Quick start
- Rename
.env.example
to.env
and fill it with your environment values. - Install Docker and migrate tool for applying migrations.
- Run project by this command:
make docker.run
# Process:
# - Generate API docs by Swagger
# - Create a new Docker network for containers
# - Build and run Docker containers (Fiber, PostgreSQL)
# - Apply database migrations (using github.com/golang-migrate/migrate)
- Go to your API Docs page: 127.0.0.1:5000/swagger/index.html
P.S.
If you want more articles like this on this blog, then post a comment below and subscribe to me. Thanks! 😘
And, of course, you can support me by donating at LiberaPay. Each donation will be used to write new articles and develop non-profit open-source projects for…
My approach to Go project architecture
Over the past two years, I have tried many structures for the Go application, but settled on mine, which I'll try to explain to you now.
Folder with business logic only
./app
folder doesn't care about what database driver you're using or which caching solution your choose or any third-party things.
-
./app/controllers
folder for functional controllers (used in routes); -
./app/models
folder for describe business models and methods; -
./app/queries
folder for describe queries for models;
Folder with API Documentation
./docs
folder contains config files for auto-generated API Docs by Swagger.
Folder with project-specific functionality
./pkg
folder contains all the project-specific code tailored only for your business use case, like configs, middleware, routes or utilities.
-
./pkg/configs
folder for configuration functions; -
./pkg/middleware
folder for add middleware; -
./pkg/routes
folder for describe routes of your project; -
./pkg/utils
folder with utility functions (server starter, generators, etc);
Folder with platform-level logic
./platform
folder contains all the platform-level logic that will build up the actual project, like setting up the database or cache server instance and storing migrations.
-
./platform/database
folder with database setup functions; -
./platform/migrations
folder with migration files;
Project configuration
The config of the project may seem very complicated at first sight. Don't worry, I'll describe each point as simply and easily as possible.
Makefile
I highly recommend using a Makefile
for faster project management! But in this article, I want to show the whole process. So, I will write all commands directly, without magic make
.
👋 If you already know it, here is a link to the full project's
Makefile
.
Fiber config in ENV file
I know that some people like to use YML files to configure their Go applications, but I'm used to working with classical .env
configurations and don't see much benefit from YML (even though I wrote an article about this kind of app configuration in Go in the past).
The config file for this project will be as follows:
# ./.env
# Server settings:
SERVER_URL="0.0.0.0:5000"
SERVER_READ_TIMEOUT=60
# JWT settings:
JWT_SECRET_KEY="secret"
JWT_SECRET_KEY_EXPIRE_MINUTES_COUNT=15
# Database settings:
DB_SERVER_URL="host=localhost port=5432 user=postgres password=password dbname=postgres sslmode=disable"
DB_MAX_CONNECTIONS=100
DB_MAX_IDLE_CONNECTIONS=10
DB_MAX_LIFETIME_CONNECTIONS=2
Docker network
Install and run Docker service for your OS. By the way, in this tutorial I'm using the latest version (at this moment) v20.10.2
.
OK, let's make a new Docker network, called dev-network
:
docker network create -d bridge dev-network
We will use it in the future when we run the database and the Fiber instance in isolated containers. If this is not done, the two containers will not be able to communicate with each other.
☝️ For more information, please visit: https://docs.docker.com/network/
PostgreSQL and initial migration
So, let's start the container with the database:
docker run --rm -d \
--name dev-postgres \
--network dev-network \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=password \
-e POSTGRES_DB=postgres \
-v ${HOME}/dev-postgres/data/:/var/lib/postgresql/data \
-p 5432:5432 \
postgres
Check, if the container is running. For example, by ctop
console utility:
Great! Now we are ready to do the migration of the original structure. Here is the file for up
migration, called 000001_create_init_tables.up.sql
:
-- ./platform/migrations/000001_create_init_tables.up.sql
-- Add UUID extension
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- Set timezone
-- For more information, please visit:
-- https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
SET TIMEZONE="Europe/Moscow";
-- Create books table
CREATE TABLE books (
id UUID DEFAULT uuid_generate_v4 () PRIMARY KEY,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW (),
updated_at TIMESTAMP NULL,
title VARCHAR (255) NOT NULL,
author VARCHAR (255) NOT NULL,
book_status INT NOT NULL,
book_attrs JSONB NOT NULL
);
-- Add indexes
CREATE INDEX active_books ON books (title) WHERE book_status = 1;
☝️ For easily working with an additional book attributes, I use
JSONB
type for abook_attrs
field. For more information, please visit PostgreSQL docs.
And 000001_create_init_tables.down.sql
for down
this migration:
-- ./platform/migrations/000001_create_init_tables.down.sql
-- Delete tables
DROP TABLE IF EXISTS books;
Okay! We can roll this migration.
👍 I recommend to use golang-migrate/migrate tool for easily up & down your database migrations in one console command.
migrate \
-path $(PWD)/platform/migrations \
-database "postgres://postgres:password@localhost/postgres?sslmode=disable" \
up
Dockerfile for the Fiber app
Create a Dockerfile
in the project root folder:
# ./Dockerfile
FROM golang:1.16-alpine AS builder
# Move to working directory (/build).
WORKDIR /build
# Copy and download dependency using go mod.
COPY go.mod go.sum ./
RUN go mod download
# Copy the code into the container.
COPY . .
# Set necessary environment variables needed for our image
# and build the API server.
ENV CGO_ENABLED=0 GOOS=linux GOARCH=amd64
RUN go build -ldflags="-s -w" -o apiserver .
FROM scratch
# Copy binary and config files from /build
# to root folder of scratch container.
COPY --from=builder ["/build/apiserver", "/build/.env", "/"]
# Export necessary port.
EXPOSE 5000
# Command to run when starting the container.
ENTRYPOINT ["/apiserver"]
Yes, I'm using two-staged container build and Golang 1.16.x
. App will be build with CGO_ENABLED=0
and -ldflags="-s -w"
to reduce size of the finished binary. Otherwise, this is the most common Dockerfile
for any Go project, that you can use anywhere.
Command to build the Fiber Docker image:
docker build -t fiber .
☝️ Don't forget to add
.dockerignore
file to the project's root folder with all files and folders, which should be ignored when creating a container. Here is an example, what I'm using in this tutorial.
Command to create and start container from image:
docker run --rm -d \
--name dev-fiber \
--network dev-network \
-p 5000:5000 \
fiber
Swagger
As you can guess from the title, we're not going to worry too much about documenting our API methods. Simply because there is a great tool like Swagger that will do all the work for us!
- swaggo/swag package for easily generate Swagger config in Go;
- arsmn/fiber-swagger official Fiber's middleware;
Practical part
Well, we have prepared all the necessary configuration files and the working environment, and we know what we are going to create. Now it's time to open our favorite IDE and start writing code.
👋 Be aware, because I will be explaining some points directly in the comments in the code, not in the article.
Create a model
Before implementing a model, I always create a migration file with an SQL structure (from the Chapter 3). This makes it much easier to present all the necessary model fields at once.
// ./app/models/book_model.go
package models
import (
"database/sql/driver"
"encoding/json"
"errors"
"time"
"github.com/google/uuid"
)
// Book struct to describe book object.
type Book struct {
ID uuid.UUID `db:"id" json:"id" validate:"required,uuid"`
CreatedAt time.Time `db:"created_at" json:"created_at"`
UpdatedAt time.Time `db:"updated_at" json:"updated_at"`
UserID uuid.UUID `db:"user_id" json:"user_id" validate:"required,uuid"`
Title string `db:"title" json:"title" validate:"required,lte=255"`
Author string `db:"author" json:"author" validate:"required,lte=255"`
BookStatus int `db:"book_status" json:"book_status" validate:"required,len=1"`
BookAttrs BookAttrs `db:"book_attrs" json:"book_attrs" validate:"required,dive"`
}
// BookAttrs struct to describe book attributes.
type BookAttrs struct {
Picture string `json:"picture"`
Description string `json:"description"`
Rating int `json:"rating" validate:"min=1,max=10"`
}
// ...
👍 I recommend to use the google/uuid package to create unique IDs, because this is a more versatile way to protect your application against common number brute force attacks. Especially if your REST API will have public methods without authorization and request limit.
But that's not all. You need to write two special methods:
-
Value()
, for return a JSON-encoded representation of the struct; -
Scan()
, for decode a JSON-encoded value into the struct fields;
They might look like this:
// ...
// Value make the BookAttrs struct implement the driver.Valuer interface.
// This method simply returns the JSON-encoded representation of the struct.
func (b BookAttrs) Value() (driver.Value, error) {
return json.Marshal(b)
}
// Scan make the BookAttrs struct implement the sql.Scanner interface.
// This method simply decodes a JSON-encoded value into the struct fields.
func (b *BookAttrs) Scan(value interface{}) error {
j, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
return json.Unmarshal(j, &b)
}
Create validators for a model fields
Okay, let's define the fields we need to check on the input before passing them to the controller business logic:
-
ID
field, for checking a valid UUID;
These fields are the biggest concern, because in some scenarios they will come to us from users. By the way, that's why we not only validate them, but consider them required
.
And this is how I implement the validator:
// ./app/utils/validator.go
package utils
import (
"github.com/go-playground/validator/v10"
"github.com/google/uuid"
)
// NewValidator func for create a new validator for model fields.
func NewValidator() *validator.Validate {
// Create a new validator for a Book model.
validate := validator.New()
// Custom validation for uuid.UUID fields.
_ = validate.RegisterValidation("uuid", func(fl validator.FieldLevel) bool {
field := fl.Field().String()
if _, err := uuid.Parse(field); err != nil {
return true
}
return false
})
return validate
}
// ValidatorErrors func for show validation errors for each invalid fields.
func ValidatorErrors(err error) map[string]string {
// Define fields map.
fields := map[string]string{}
// Make error message for each invalid field.
for _, err := range err.(validator.ValidationErrors) {
fields[err.Field()] = err.Error()
}
return fields
}
👌 I use go-playground/validator
v10
for release this feature.
Create queries and controllers
Database queries
So as not to lose performance, I like to work with pure SQL queries without sugar, like gorm
or similar packages. It gives a much better understanding of how the application works, which will help in the future not to make silly mistakes, when optimizing database queries!
// ./app/queries/book_query.go
package queries
import (
"github.com/google/uuid"
"github.com/jmoiron/sqlx"
"github.com/koddr/tutorial-go-fiber-rest-api/app/models"
)
// BookQueries struct for queries from Book model.
type BookQueries struct {
*sqlx.DB
}
// GetBooks method for getting all books.
func (q *BookQueries) GetBooks() ([]models.Book, error) {
// Define books variable.
books := []models.Book{}
// Define query string.
query := `SELECT * FROM books`
// Send query to database.
err := q.Get(&books, query)
if err != nil {
// Return empty object and error.
return books, err
}
// Return query result.
return books, nil
}
// GetBook method for getting one book by given ID.
func (q *BookQueries) GetBook(id uuid.UUID) (models.Book, error) {
// Define book variable.
book := models.Book{}
// Define query string.
query := `SELECT * FROM books WHERE id = $1`
// Send query to database.
err := q.Get(&book, query, id)
if err != nil {
// Return empty object and error.
return book, err
}
// Return query result.
return book, nil
}
// CreateBook method for creating book by given Book object.
func (q *BookQueries) CreateBook(b *models.Book) error {
// Define query string.
query := `INSERT INTO books VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`
// Send query to database.
_, err := q.Exec(query, b.ID, b.CreatedAt, b.UpdatedAt, b.UserID, b.Title, b.Author, b.BookStatus, b.BookAttrs)
if err != nil {
// Return only error.
return err
}
// This query returns nothing.
return nil
}
// UpdateBook method for updating book by given Book object.
func (q *BookQueries) UpdateBook(id uuid.UUID, b *models.Book) error {
// Define query string.
query := `UPDATE books SET updated_at = $2, title = $3, author = $4, book_status = $5, book_attrs = $6 WHERE id = $1`
// Send query to database.
_, err := q.Exec(query, id, b.UpdatedAt, b.Title, b.Author, b.BookStatus, b.BookAttrs)
if err != nil {
// Return only error.
return err
}
// This query returns nothing.
return nil
}
// DeleteBook method for delete book by given ID.
func (q *BookQueries) DeleteBook(id uuid.UUID) error {
// Define query string.
query := `DELETE FROM books WHERE id = $1`
// Send query to database.
_, err := q.Exec(query, id)
if err != nil {
// Return only error.
return err
}
// This query returns nothing.
return nil
}
Create model controllers
The principle of the GET
methods:
- Make request to the API endpoint;
- Make a connection to the database (or an error);
- Make a query to get record(s) from the table
books
(or an error); - Return the status
200
and JSON with a founded book(s);
// ./app/controllers/book_controller.go
package controllers
import (
"time"
"github.com/gofiber/fiber/v2"
"github.com/google/uuid"
"github.com/koddr/tutorial-go-fiber-rest-api/app/models"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/utils"
"github.com/koddr/tutorial-go-fiber-rest-api/platform/database"
)
// GetBooks func gets all exists books.
// @Description Get all exists books.
// @Summary get all exists books
// @Tags Books
// @Accept json
// @Produce json
// @Success 200 {array} models.Book
// @Router /v1/books [get]
func GetBooks(c *fiber.Ctx) error {
// Create database connection.
db, err := database.OpenDBConnection()
if err != nil {
// Return status 500 and database connection error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Get all books.
books, err := db.GetBooks()
if err != nil {
// Return, if books not found.
return c.Status(fiber.StatusNotFound).JSON(fiber.Map{
"error": true,
"msg": "books were not found",
"count": 0,
"books": nil,
})
}
// Return status 200 OK.
return c.JSON(fiber.Map{
"error": false,
"msg": nil,
"count": len(books),
"books": books,
})
}
// GetBook func gets book by given ID or 404 error.
// @Description Get book by given ID.
// @Summary get book by given ID
// @Tags Book
// @Accept json
// @Produce json
// @Param id path string true "Book ID"
// @Success 200 {object} models.Book
// @Router /v1/book/{id} [get]
func GetBook(c *fiber.Ctx) error {
// Catch book ID from URL.
id, err := uuid.Parse(c.Params("id"))
if err != nil {
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Create database connection.
db, err := database.OpenDBConnection()
if err != nil {
// Return status 500 and database connection error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Get book by ID.
book, err := db.GetBook(id)
if err != nil {
// Return, if book not found.
return c.Status(fiber.StatusNotFound).JSON(fiber.Map{
"error": true,
"msg": "book with the given ID is not found",
"book": nil,
})
}
// Return status 200 OK.
return c.JSON(fiber.Map{
"error": false,
"msg": nil,
"book": book,
})
}
// ...
The principle of the POST
methods:
- Make a request to the API endpoint;
- Check, if request
Header
has a valid JWT; - Check, if expire date from JWT greather than now (or an error);
- Parse Body of request and bind fields to the Book struct (or an error);
- Make a connection to the database (or an error);
- Validate struct fields with a new content from Body (or an error);
- Make a query to create a new record in the table
books
(or an error); - Return the status
200
and JSON with a new book;
// ...
// CreateBook func for creates a new book.
// @Description Create a new book.
// @Summary create a new book
// @Tags Book
// @Accept json
// @Produce json
// @Param title body string true "Title"
// @Param author body string true "Author"
// @Param book_attrs body models.BookAttrs true "Book attributes"
// @Success 200 {object} models.Book
// @Security ApiKeyAuth
// @Router /v1/book [post]
func CreateBook(c *fiber.Ctx) error {
// Get now time.
now := time.Now().Unix()
// Get claims from JWT.
claims, err := utils.ExtractTokenMetadata(c)
if err != nil {
// Return status 500 and JWT parse error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Set expiration time from JWT data of current book.
expires := claims.Expires
// Checking, if now time greather than expiration from JWT.
if now > expires {
// Return status 401 and unauthorized error message.
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{
"error": true,
"msg": "unauthorized, check expiration time of your token",
})
}
// Create new Book struct
book := &models.Book{}
// Check, if received JSON data is valid.
if err := c.BodyParser(book); err != nil {
// Return status 400 and error message.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Create database connection.
db, err := database.OpenDBConnection()
if err != nil {
// Return status 500 and database connection error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Create a new validator for a Book model.
validate := utils.NewValidator()
// Set initialized default data for book:
book.ID = uuid.New()
book.CreatedAt = time.Now()
book.BookStatus = 1 // 0 == draft, 1 == active
// Validate book fields.
if err := validate.Struct(book); err != nil {
// Return, if some fields are not valid.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": utils.ValidatorErrors(err),
})
}
// Delete book by given ID.
if err := db.CreateBook(book); err != nil {
// Return status 500 and error message.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Return status 200 OK.
return c.JSON(fiber.Map{
"error": false,
"msg": nil,
"book": book,
})
}
// ...
The principle of the PUT
methods:
- Make a request to the API endpoint;
- Check, if request
Header
has a valid JWT; - Check, if expire date from JWT greather than now (or an error);
- Parse Body of request and bind fields to the Book struct (or an error);
- Make a connection to the database (or an error);
- Validate struct fields with a new content from Body (or an error);
- Check, if book with this ID is exists (or an error);
- Make a query to update this record in the table
books
(or an error); - Return the status
201
without content;
// ...
// UpdateBook func for updates book by given ID.
// @Description Update book.
// @Summary update book
// @Tags Book
// @Accept json
// @Produce json
// @Param id body string true "Book ID"
// @Param title body string true "Title"
// @Param author body string true "Author"
// @Param book_status body integer true "Book status"
// @Param book_attrs body models.BookAttrs true "Book attributes"
// @Success 201 {string} status "ok"
// @Security ApiKeyAuth
// @Router /v1/book [put]
func UpdateBook(c *fiber.Ctx) error {
// Get now time.
now := time.Now().Unix()
// Get claims from JWT.
claims, err := utils.ExtractTokenMetadata(c)
if err != nil {
// Return status 500 and JWT parse error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Set expiration time from JWT data of current book.
expires := claims.Expires
// Checking, if now time greather than expiration from JWT.
if now > expires {
// Return status 401 and unauthorized error message.
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{
"error": true,
"msg": "unauthorized, check expiration time of your token",
})
}
// Create new Book struct
book := &models.Book{}
// Check, if received JSON data is valid.
if err := c.BodyParser(book); err != nil {
// Return status 400 and error message.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Create database connection.
db, err := database.OpenDBConnection()
if err != nil {
// Return status 500 and database connection error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Checking, if book with given ID is exists.
foundedBook, err := db.GetBook(book.ID)
if err != nil {
// Return status 404 and book not found error.
return c.Status(fiber.StatusNotFound).JSON(fiber.Map{
"error": true,
"msg": "book with this ID not found",
})
}
// Set initialized default data for book:
book.UpdatedAt = time.Now()
// Create a new validator for a Book model.
validate := utils.NewValidator()
// Validate book fields.
if err := validate.Struct(book); err != nil {
// Return, if some fields are not valid.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": utils.ValidatorErrors(err),
})
}
// Update book by given ID.
if err := db.UpdateBook(foundedBook.ID, book); err != nil {
// Return status 500 and error message.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Return status 201.
return c.SendStatus(fiber.StatusCreated)
}
// ...
The principle of the DELETE
methods:
- Make a request to the API endpoint;
- Check, if request
Header
has a valid JWT; - Check, if expire date from JWT greather than now (or an error);
- Parse Body of request and bind fields to the Book struct (or an error);
- Make a connection to the database (or an error);
- Validate struct fields with a new content from Body (or an error);
- Check, if book with this ID is exists (or an error);
- Make a query to delete this record from the table
books
(or an error); - Return the status
204
without content;
// ...
// DeleteBook func for deletes book by given ID.
// @Description Delete book by given ID.
// @Summary delete book by given ID
// @Tags Book
// @Accept json
// @Produce json
// @Param id body string true "Book ID"
// @Success 204 {string} status "ok"
// @Security ApiKeyAuth
// @Router /v1/book [delete]
func DeleteBook(c *fiber.Ctx) error {
// Get now time.
now := time.Now().Unix()
// Get claims from JWT.
claims, err := utils.ExtractTokenMetadata(c)
if err != nil {
// Return status 500 and JWT parse error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Set expiration time from JWT data of current book.
expires := claims.Expires
// Checking, if now time greather than expiration from JWT.
if now > expires {
// Return status 401 and unauthorized error message.
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{
"error": true,
"msg": "unauthorized, check expiration time of your token",
})
}
// Create new Book struct
book := &models.Book{}
// Check, if received JSON data is valid.
if err := c.BodyParser(book); err != nil {
// Return status 400 and error message.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Create a new validator for a Book model.
validate := utils.NewValidator()
// Validate only one book field ID.
if err := validate.StructPartial(book, "id"); err != nil {
// Return, if some fields are not valid.
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": utils.ValidatorErrors(err),
})
}
// Create database connection.
db, err := database.OpenDBConnection()
if err != nil {
// Return status 500 and database connection error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Checking, if book with given ID is exists.
foundedBook, err := db.GetBook(book.ID)
if err != nil {
// Return status 404 and book not found error.
return c.Status(fiber.StatusNotFound).JSON(fiber.Map{
"error": true,
"msg": "book with this ID not found",
})
}
// Delete book by given ID.
if err := db.DeleteBook(foundedBook.ID); err != nil {
// Return status 500 and error message.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Return status 204 no content.
return c.SendStatus(fiber.StatusNoContent)
}
Method for get a new Access token (JWT)
- Make request to the API endpoint;
- Return the status
200
and JSON with a new access token;
// ./app/controllers/token_controller.go
package controllers
import (
"github.com/gofiber/fiber/v2"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/utils"
)
// GetNewAccessToken method for create a new access token.
// @Description Create a new access token.
// @Summary create a new access token
// @Tags Token
// @Accept json
// @Produce json
// @Success 200 {string} status "ok"
// @Router /v1/token/new [get]
func GetNewAccessToken(c *fiber.Ctx) error {
// Generate a new Access token.
token, err := utils.GenerateNewAccessToken()
if err != nil {
// Return status 500 and token generation error.
return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
return c.JSON(fiber.Map{
"error": false,
"msg": nil,
"access_token": token,
})
}
The main function
This is the most important feature in our entire application. It loads the configuration from the .env
file, defines the Swagger settings, creates a new Fiber instance, connects the necessary groups of endpoints and starts the API server.
// ./main.go
package main
import (
"github.com/gofiber/fiber/v2"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/configs"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/middleware"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/routes"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/utils"
_ "github.com/joho/godotenv/autoload" // load .env file automatically
_ "github.com/koddr/tutorial-go-fiber-rest-api/docs" // load API Docs files (Swagger)
)
// @title API
// @version 1.0
// @description This is an auto-generated API Docs.
// @termsOfService http://swagger.io/terms/
// @contact.name API Support
// @contact.email your@mail.com
// @license.name Apache 2.0
// @license.url http://www.apache.org/licenses/LICENSE-2.0.html
// @securityDefinitions.apikey ApiKeyAuth
// @in header
// @name Authorization
// @BasePath /api
func main() {
// Define Fiber config.
config := configs.FiberConfig()
// Define a new Fiber app with config.
app := fiber.New(config)
// Middlewares.
middleware.FiberMiddleware(app) // Register Fiber's middleware for app.
// Routes.
routes.SwaggerRoute(app) // Register a route for API Docs (Swagger).
routes.PublicRoutes(app) // Register a public routes for app.
routes.PrivateRoutes(app) // Register a private routes for app.
routes.NotFoundRoute(app) // Register route for 404 Error.
// Start server (with graceful shutdown).
utils.StartServerWithGracefulShutdown(app)
}
A middleware functions
Since in this application I want to show how to use JWT to authorize some queries, we need to write additional middleware to validate it:
// ./pkg/middleware/jwt_middleware.go
package middleware
import (
"os"
"github.com/gofiber/fiber/v2"
jwtMiddleware "github.com/gofiber/jwt/v2"
)
// JWTProtected func for specify routes group with JWT authentication.
// See: https://github.com/gofiber/jwt
func JWTProtected() func(*fiber.Ctx) error {
// Create config for JWT authentication middleware.
config := jwtMiddleware.Config{
SigningKey: []byte(os.Getenv("JWT_SECRET_KEY")),
ContextKey: "jwt", // used in private routes
ErrorHandler: jwtError,
}
return jwtMiddleware.New(config)
}
func jwtError(c *fiber.Ctx, err error) error {
// Return status 401 and failed authentication error.
if err.Error() == "Missing or malformed JWT" {
return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
// Return status 401 and failed authentication error.
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{
"error": true,
"msg": err.Error(),
})
}
Routes for the API endpoints
- For public methods:
// ./pkg/routes/private_routes.go
package routes
import (
"github.com/gofiber/fiber/v2"
"github.com/koddr/tutorial-go-fiber-rest-api/app/controllers"
)
// PublicRoutes func for describe group of public routes.
func PublicRoutes(a *fiber.App) {
// Create routes group.
route := a.Group("/api/v1")
// Routes for GET method:
route.Get("/books", controllers.GetBooks) // get list of all books
route.Get("/book/:id", controllers.GetBook) // get one book by ID
route.Get("/token/new", controllers.GetNewAccessToken) // create a new access tokens
}
- For private (JWT protected) methods:
// ./pkg/routes/private_routes.go
package routes
import (
"github.com/gofiber/fiber/v2"
"github.com/koddr/tutorial-go-fiber-rest-api/app/controllers"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/middleware"
)
// PrivateRoutes func for describe group of private routes.
func PrivateRoutes(a *fiber.App) {
// Create routes group.
route := a.Group("/api/v1")
// Routes for POST method:
route.Post("/book", middleware.JWTProtected(), controllers.CreateBook) // create a new book
// Routes for PUT method:
route.Put("/book", middleware.JWTProtected(), controllers.UpdateBook) // update one book by ID
// Routes for DELETE method:
route.Delete("/book", middleware.JWTProtected(), controllers.DeleteBook) // delete one book by ID
}
- For Swagger:
// ./pkg/routes/swagger_route.go
package routes
import (
"github.com/gofiber/fiber/v2"
swagger "github.com/arsmn/fiber-swagger/v2"
)
// SwaggerRoute func for describe group of API Docs routes.
func SwaggerRoute(a *fiber.App) {
// Create routes group.
route := a.Group("/swagger")
// Routes for GET method:
route.Get("*", swagger.Handler) // get one user by ID
}
-
Not found
(404) route:
// ./pkg/routes/not_found_route.go
package routes
import "github.com/gofiber/fiber/v2"
// NotFoundRoute func for describe 404 Error route.
func NotFoundRoute(a *fiber.App) {
// Register new special route.
a.Use(
// Anonimus function.
func(c *fiber.Ctx) error {
// Return HTTP 404 status and JSON response.
return c.Status(fiber.StatusNotFound).JSON(fiber.Map{
"error": true,
"msg": "sorry, endpoint is not found",
})
},
)
}
Database connection
The database connection is the most important part of this application (as well as any other, to be honest). I like to break this process down into two parts.
- The method for the connection:
// ./platform/database/open_db_connection.go
package database
import "github.com/koddr/tutorial-go-fiber-rest-api/app/queries"
// Queries struct for collect all app queries.
type Queries struct {
*queries.BookQueries // load queries from Book model
}
// OpenDBConnection func for opening database connection.
func OpenDBConnection() (*Queries, error) {
// Define a new PostgreSQL connection.
db, err := PostgreSQLConnection()
if err != nil {
return nil, err
}
return &Queries{
// Set queries from models:
BookQueries: &queries.BookQueries{DB: db}, // from Book model
}, nil
}
- The specific connection settings for the selected database:
// ./platform/database/postgres.go
package database
import (
"fmt"
"os"
"strconv"
"time"
"github.com/jmoiron/sqlx"
_ "github.com/jackc/pgx/v4/stdlib" // load pgx driver for PostgreSQL
)
// PostgreSQLConnection func for connection to PostgreSQL database.
func PostgreSQLConnection() (*sqlx.DB, error) {
// Define database connection settings.
maxConn, _ := strconv.Atoi(os.Getenv("DB_MAX_CONNECTIONS"))
maxIdleConn, _ := strconv.Atoi(os.Getenv("DB_MAX_IDLE_CONNECTIONS"))
maxLifetimeConn, _ := strconv.Atoi(os.Getenv("DB_MAX_LIFETIME_CONNECTIONS"))
// Define database connection for PostgreSQL.
db, err := sqlx.Connect("pgx", os.Getenv("DB_SERVER_URL"))
if err != nil {
return nil, fmt.Errorf("error, not connected to database, %w", err)
}
// Set database connection settings.
db.SetMaxOpenConns(maxConn) // the default is 0 (unlimited)
db.SetMaxIdleConns(maxIdleConn) // defaultMaxIdleConns = 2
db.SetConnMaxLifetime(time.Duration(maxLifetimeConn)) // 0, connections are reused forever
// Try to ping database.
if err := db.Ping(); err != nil {
defer db.Close() // close database connection
return nil, fmt.Errorf("error, not sent ping to database, %w", err)
}
return db, nil
}
☝️ This approach helps to connect additional databases more easily if required and always keep a clear hierarchy of data storage in the application.
Useful utilities
- For start API server (with a graceful shutdown or simple for dev):
// ./pkg/utils/start_server.go
package utils
import (
"log"
"os"
"os/signal"
"github.com/gofiber/fiber/v2"
)
// StartServerWithGracefulShutdown function for starting server with a graceful shutdown.
func StartServerWithGracefulShutdown(a *fiber.App) {
// Create channel for idle connections.
idleConnsClosed := make(chan struct{})
go func() {
sigint := make(chan os.Signal, 1)
signal.Notify(sigint, os.Interrupt) // Catch OS signals.
<-sigint
// Received an interrupt signal, shutdown.
if err := a.Shutdown(); err != nil {
// Error from closing listeners, or context timeout:
log.Printf("Oops... Server is not shutting down! Reason: %v", err)
}
close(idleConnsClosed)
}()
// Run server.
if err := a.Listen(os.Getenv("SERVER_URL")); err != nil {
log.Printf("Oops... Server is not running! Reason: %v", err)
}
<-idleConnsClosed
}
// StartServer func for starting a simple server.
func StartServer(a *fiber.App) {
// Run server.
if err := a.Listen(os.Getenv("SERVER_URL")); err != nil {
log.Printf("Oops... Server is not running! Reason: %v", err)
}
}
- For generate a valid JWT:
// ./pkg/utils/jwt_generator.go
package utils
import (
"os"
"strconv"
"time"
"github.com/golang-jwt/jwt"
)
// GenerateNewAccessToken func for generate a new Access token.
func GenerateNewAccessToken() (string, error) {
// Set secret key from .env file.
secret := os.Getenv("JWT_SECRET_KEY")
// Set expires minutes count for secret key from .env file.
minutesCount, _ := strconv.Atoi(os.Getenv("JWT_SECRET_KEY_EXPIRE_MINUTES_COUNT"))
// Create a new claims.
claims := jwt.MapClaims{}
// Set public claims:
claims["exp"] = time.Now().Add(time.Minute * time.Duration(minutesCount)).Unix()
// Create a new JWT access token with claims.
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
// Generate token.
t, err := token.SignedString([]byte(secret))
if err != nil {
// Return error, it JWT token generation failed.
return "", err
}
return t, nil
}
- For parse and validate JWT:
// ./pkg/utils/jwt_parser.go
package utils
import (
"os"
"strings"
"github.com/golang-jwt/jwt"
"github.com/gofiber/fiber/v2"
)
// TokenMetadata struct to describe metadata in JWT.
type TokenMetadata struct {
Expires int64
}
// ExtractTokenMetadata func to extract metadata from JWT.
func ExtractTokenMetadata(c *fiber.Ctx) (*TokenMetadata, error) {
token, err := verifyToken(c)
if err != nil {
return nil, err
}
// Setting and checking token and credentials.
claims, ok := token.Claims.(jwt.MapClaims)
if ok && token.Valid {
// Expires time.
expires := int64(claims["exp"].(float64))
return &TokenMetadata{
Expires: expires,
}, nil
}
return nil, err
}
func extractToken(c *fiber.Ctx) string {
bearToken := c.Get("Authorization")
// Normally Authorization HTTP header.
onlyToken := strings.Split(bearToken, " ")
if len(onlyToken) == 2 {
return onlyToken[1]
}
return ""
}
func verifyToken(c *fiber.Ctx) (*jwt.Token, error) {
tokenString := extractToken(c)
token, err := jwt.Parse(tokenString, jwtKeyFunc)
if err != nil {
return nil, err
}
return token, nil
}
func jwtKeyFunc(token *jwt.Token) (interface{}, error) {
return []byte(os.Getenv("JWT_SECRET_KEY")), nil
}
Testing the application
So, we're getting to the most important stage! Let's check our Fiber application through testing. I'll show you the principle by testing private routes (JWT protected).
☝️ As always, I will use Fiber's built-in
Test()
method and an awesome package stretchr/testify for testing Golang apps.
Also, I like to put the configuration for testing in a separate file, I don't want to mix a production config with a test config. So, I use the file called .env.test
, which I will add to the root of the project.
Pay attention to the part of the code where routes are defined. We're calling the real routes of our application, so before running the test, you need to bring up the database (e.g. in a Docker container for simplicity).
// ./pkg/routes/private_routes_test.go
package routes
import (
"io"
"net/http/httptest"
"strings"
"testing"
"github.com/gofiber/fiber/v2"
"github.com/joho/godotenv"
"github.com/koddr/tutorial-go-fiber-rest-api/pkg/utils"
"github.com/stretchr/testify/assert"
)
func TestPrivateRoutes(t *testing.T) {
// Load .env.test file from the root folder.
if err := godotenv.Load("../../.env.test"); err != nil {
panic(err)
}
// Create a sample data string.
dataString := `{"id": "00000000-0000-0000-0000-000000000000"}`
// Create access token.
token, err := utils.GenerateNewAccessToken()
if err != nil {
panic(err)
}
// Define a structure for specifying input and output data of a single test case.
tests := []struct {
description string
route string // input route
method string // input method
tokenString string // input token
body io.Reader
expectedError bool
expectedCode int
}{
{
description: "delete book without JWT and body",
route: "/api/v1/book",
method: "DELETE",
tokenString: "",
body: nil,
expectedError: false,
expectedCode: 400,
},
{
description: "delete book without right credentials",
route: "/api/v1/book",
method: "DELETE",
tokenString: "Bearer " + token,
body: strings.NewReader(dataString),
expectedError: false,
expectedCode: 403,
},
{
description: "delete book with credentials",
route: "/api/v1/book",
method: "DELETE",
tokenString: "Bearer " + token,
body: strings.NewReader(dataString),
expectedError: false,
expectedCode: 404,
},
}
// Define a new Fiber app.
app := fiber.New()
// Define routes.
PrivateRoutes(app)
// Iterate through test single test cases
for _, test := range tests {
// Create a new http request with the route from the test case.
req := httptest.NewRequest(test.method, test.route, test.body)
req.Header.Set("Authorization", test.tokenString)
req.Header.Set("Content-Type", "application/json")
// Perform the request plain with the app.
resp, err := app.Test(req, -1) // the -1 disables request latency
// Verify, that no error occurred, that is not expected
assert.Equalf(t, test.expectedError, err != nil, test.description)
// As expected errors lead to broken responses,
// the next test case needs to be processed.
if test.expectedError {
continue
}
// Verify, if the status code is as expected.
assert.Equalf(t, test.expectedCode, resp.StatusCode, test.description)
}
}
// ...
Run project locally
Let's run Docker containers, apply migrations and go to http://127.0.0.1:5000/swagger/index.html
:
It works. Woohoo! 🎉
A self-check block of knowledge
Try not to peek at the text of the tutorial and answer as quickly and honestly as possible. Don't worry if you forgot something! Just keep going:
- What is the name of the technology that allows applications to run in an isolated environment?
- Where should the application's business logic be located (folder name)?
- What file should be created in the root of the project to describe the process of creating the container for the application?
- What is UUID and why do we use it for ID?
- What type of PostgreSQL field did we use to create the model for book attributes?
- Why is it better to use pure SQL in Go apps?
- Where do you need to describe the API method for auto-generating documentation (by Swagger)?
- Why separate configurations in testing?
Plan for further development
For further (independent) development of this application, I recommend considering the following options:
- Upgrade the
CreateBook
method: add a handler to save picture to a cloud storage service (e.g., Amazon S3 or similar) and save only picture ID to our database; - Upgrade the
GetBook
andGetBooks
methods: add a handler to change picture ID from a cloud service to direct link to this picture; - Add a new method for registering new users (e.g., registered users can get a role, which will allow them to perform some methods in the REST API);
- Add a new method for user authorization (e.g., after authorization, users receive a JWT token that contains credentials according to its role);
- Add a standalone container with Redis (or similar) to store the sessions of these authorized users;
Photos by
- Joshua Woroniecki https://unsplash.com/photos/LeleZeefJ7M
- Maxwell Nelson https://unsplash.com/photos/Y9w872CNIyI
- Vic Shóstak https://shostak.dev
P.S.
If you want more articles (like this) on this blog, then post a comment below and subscribe to me. Thanks! 😻
❗️ You can support me on Boosty, both on a permanent and on a one-time basis. All proceeds from this way will go to support my OSS projects and will energize me to create new products and articles for the community.
And of course, you can help me make developers' lives even better! Just connect to one of my projects as a contributor. It's easy!
My main projects that need your help (and stars) 👇
- 🔥 gowebly: A next-generation CLI tool that makes it easy to create amazing web applications with Go on the backend, using htmx, hyperscript or Alpine.js and the most popular CSS frameworks on the frontend.
- ✨ create-go-app: Create a new production-ready project with Go backend, frontend and deploy automation by running one CLI command.
Top comments (20)
Thank you so much for taking the time to make this tutorial! 😄
Really good tutorial. Only one complain.
You should never, ever mix the database layer with http layer. Or any other layer.
db:"id" json:"id"
<---- never do this.Data that you use in JSON "objects" should never be mixed with Models "objects"
Those 2 things work in parallel universes, they should never have contact with each other. One thing is what an frontend application sends to you, another thing is what you put or get from the databases and another thing is what you return from the database and another thing is what you return to the user.
Except on really basic cases, like Todos and Hello worlds that data is always diferent. I advise to take a look at this awesome post:
eltonminetto.dev/en/post/2020-07-0...
It explains in a lot more detail. Of course this does not affect simple applications. But as soon as you start having a few more complexities in your code, those "structs" will fail to provide you what you need.
A more concrete example of what I'm trying to explain would be:
if you signin up a user using email, password. would you use the same "struct" to map it to the database?
a struct for the payload would be something like:
Does this has any relationship to what what you have in your "user(s)" table? Most likely your "user(s)" table has 15 fields.
Dont mix those.
You service would be much cleaner if you pass:
MyAuthService(ctx context.Context, dto SignInDTO)
instead of
MyAuthService(ctx context.Context, email, password string)
Hi,
This is true for models like authorization or transactional models, where you really need to separate the JSON output through the REST API (as if for the user) and the database query layer (as if our internal).
For example, if we have a
User
structure where we define what will be output in the profile on the frontend, then it's logical that for the authorization process itself we would need to make a separateUserSignIn
structure with just two fields of email and password. That's the only data we would go with a query to the database.That's where I agree.
But the model in question in the article is a common example of outputting data "as is" after the query. So, I don't really understand why to break DRY and duplicate exactly the same structure in the database layer, if they are identical and were described only to show the principle of work itself?
Especially since there is no consensus on "how to do the structure correctly for Go projects". I adhere to the principle as described in this article, for such "dumb" models, so far I have not encountered a single problem in years of working with Go 🤷
I, by the way, will describe all such things "from the real world of Go development" in detail in my new series of articles Go Fiber by Examples on this blog.
Thanks for the comment, anyway! 👍
Another useful thing from dtos:
Great tutorial! Like your style, "less water" )).
I got some issues during
migrate
installation:I have go version go1.17.1 and linux/amd64 (Arch linux)
BTW I've fixed that by building it from the source.
Just thoughts... I guess you can simplify "Docker related steps" by using
docker-compose
.Hi,
Thanks for reply!
If you have some problems with local installation
golang-migrate/migrate
, try Docker-way of them, like this:It's cool that you noticed this, but I did it on purpose to show the full setup for beginners... 😉 Also, I personally like to use Ansible playbooks instead of docker-compose to deploy the project to live servers (if that's what you want to roll out), but that would be too much for an already large tutorial.
Thank you Vic! Gonna try that.
the migrate cli command is giving me an error,
here is the command and the error
migrate -path $(pwd)/platform/migrations -database "postgres://postgres:password@localhost/postgres?sslmode=disable" up
error: dial tcp [::1]:5432: connect: connection refused
when i leave $(PWD) in uppercase i get:
bash: PWD: command not found
error: open /platform/migrations: no such file or directory
you need change localhost to dev-postgres when use docker
Hi,
What OS/platform do you use for run? Are you sure, that Docker container with PostgreSQL is up on this port and on the same Docker network?
I'm using parrot os/linux-distro. I somehow cant get to install migrate with the go toolchain, it gives me an error:
go get: module github.com/golang-migrate/migrate@upgrade found (v3.5.4+incompatible), but does not contain package github.com/golang-migrate/migrate/cmd/migrate
the unversioned command to install couldn't work too.
so i used the alternative docker run -v ... usage command to migrate, and it worked. i suggest i will just stick to using migrations with docker.
Thank you though.
Hi PHTremor,
Do you mind to share your command on the docker run -v ...usage?
I use this command and it cannot find the folder.
Thanks, Simon
Hi PHTremor,
Never mind. I got it working by using the "migrate" command as stated by Vic.
migrate \
-path $(PWD)/platform/migrations \
-database "postgres://postgres:password@localhost/postgres?sslmode=disable" \
up
I have 2 questions:
How can I test the links - creating a new book - using postman? I am generating new tokens, but failing to create a new book, get it, and deleting
How can i perform the tests - the testify - implementation; steps if possible
Hi, this article really helped me learning gofiber, but i wonder about where to store my jwt key after it generated. First, i want to create android apps and consume my go rest api. As you mentioned in the end of this article, "Add a standalone container with Redis (or similar) to store the sessions of these authorized users". So i better store the jwt into redis? What is disadvantage when i store it at fiber c.Locals or c.Session?
Very useful tutorial.
I had an erro when run make docker.run , especially in migrate.up:
If anyone can help me, thanks in advance!
Hi,
Check your password for local DB user
postgres
, as error was said:pq: password authentication failed for user "postgres"
.See
Makefile
's line 6 for more info.Hi Vic,
Sorry for a silly question. When you wrote: "Let's run Docker containers, apply migrations and go to 127.0.0.1:5000/swagger/index.html:", what are the command to "Run Docker container" and "apply migrations"?
Thanks,
Simon
Never mind! Found it in the README.md.
make docker.run
This is naming for rest api restfulapi.net/resource-naming/