Go Echo API Server Development

Create a simple api server based on a minimum configuration start project with the following functions

• db migration by sql-migrate
• db operation from apps by gorm
• Input check by go-playground/validator
• Switching of configuration files for each production and development environment
• User authentication middleware

https://github.com/nrikiji/go-echo-sample

Also, assume Firebase Authentication for user authentication and MySQL for database

What we make

Two APIs, one to retrieve a list of blog posts and the other to update posted posts. The API to list articles can be accessed by anyone, and the API to update articles can only be accessed by the person who posted the article.

Prepare

Setup

Clone the base project

$ git clone https://github.com/nrikiji/go-echo-sample

Edit database connection information to match your environment
config.yml

development:
  dialect: mysql
  datasource: root:@tcp(localhost:3306)/go-echo-example?charset=utf8mb4&collation=utf8mb4_general_ci&parseTime=true
  dir: migrations
  table: migrations
...  

posts table creation

$ sql-migrate -config config.yml create_posts

$ vi migrations/xxxxxxx-create_posts.sql
-- +migrate Up
CREATE TABLE `posts` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `user_id` bigint(20) unsigned NOT NULL,
  `title` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
  `body` text COLLATE utf8mb4_unicode_ci NOT NULL,
  PRIMARY KEY (`id`),
  FOREIGN KEY (`user_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

$ sql-migrate up -env development -config config.yml 
$ sql-migrate up -env test -config config.yml

Also, the migration file for the users table is included in the base project (simple table with only id, name and firebase_uid)

Register dummy data

Create a user with email address + password authentication from the firebase console and obtain an API key for the web (Web API key in Project Settings > General).

Also, add the private key for using Firebase Admin SDK (Firebase Admin SDK in Project Settings > Service Account) to the root of the project. (In this case, the file name is firebase_secret_key.json.

Obtain the localId (Firebase user ID) and idToken of the registered user from the API. localId is set in users.firebase_uid and idToken is set in the http header when requesting the API.

This time, request directly to firebase login API to get idToken and localId

$ curl 'https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=APIキー' \
-h 'Content-Type: application/json' \
-d '{"email": "[e-mail]", "password": "password", "returnSecureToken":true}' | jq

{
  "localId": "xxxxxxxxxxxxxxx",
  "idToken": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  }
}

Register a user in DB with the obtained localId.

insert into users (id, firebase_uid, name, created_at, updated_at) values
  (1, "xxxxxxxxxxxxxxx", "user1", now(), now());

insert into posts (user_id, title, body, created_at, updated_at) values
  (1, "title1", "body1", now(), now()), (2, "title2", "body2", now(), now());

Now we are ready for development

Implement the data manipulation part

Prepare a model that represents records retrieved from DB.

model/post.go

package model

type Post struct {
    ID uint `gorm: "primaryKey" json: "id"`
    UserID uint `json: "user_id"`
    User User `json: "user"`
    Title string `json: "title"`
    Body string `json: "body"`
}

Use gorm to add methods to the store to retrieve from and update the DB. Since we have a UserStore in the base project, we add the AllPosts and UpdatePost methods to it this time

store/post.go

package store

import (
    "errors".
    "go-echo-starter/model"

    "gorm.io/gorm"
)

func (us *UserStore) AllPosts() ([]model.Post, error) {
    var p []model.Post
    err := us.db.Preload("User").Find(&p).Error
    if err ! = nil {
        if errors.Is(err, gorm.ErrRecordNotFound) {
            return p, nil
        }
        return nil, err
    }
    return p, nil
}

func (us *UserStore) UpdatePost(post *model.Post) error {
    return us.db.Model(post).Updates(post).Error
}

Implementing the acquisition API

Implement the part that acquires the model from the store and returns the response in json when requested.

Implementation

handler/post.go

package handler

Import (
    "go-echo-starter/model" "net/http"
    "net/http"

    "github.com/labstack/echo/v4"
)

type postsResponse struct { (type postsResponse struct)
    posts []model.Post `json: "posts"`
}

func (h *Handler) getPosts(c echo.Context) error {.
    posts, err := h.userStore.AllPosts()
    if err ! = nil {
        return err

    return c.JSON(http.StatusOK, postsResponse{Posts: posts}))
}

Call the handler when a GET request is made with a path named /posts in a route

handler/routes.go

package handler

Import (
    "go-echo-starter/middleware"

    "github.com/labstack/echo/v4"
)

func (h *Handler) Register(api *echo.Group){.
    ...
    api.GET("/posts", h.getPosts)
}

Check operation

$ go run server.go
...

$ curl http://localhost:8000/api/posts | jq
{
  "posts": [
    {
      "id": 1,
      "user_id": 1,
      "user": {
        "id": 1,
        "name": "user1",
      },
      "title": "title1",
      "body": "body1",
    },
    }, "title": "title1", "body": "body1", }
}

write test

Prepare two test data with fixtures

fixtures/posts.yml

- id: 1
  user_id: 1
  title: "Title1"
  body: "Body1"

- id: 2
  user_id: 2
  title: "Title2"
  body: "Body2"

Write tests for the handler. Here we test that there is no error, and that the number of items matches.

handler/post_test.go

package handler

import (
    "encoding/json"
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/labstack/echo/v4"
    "github.com/stretchr/testify/assert"
)

func TestGetPosts(t *testing.T) {
    setup()
    req := httptest.NewRequest(echo.GET, "/api/posts", nil)
    rec := httptest.NewRecorder()

    c := e.NewContext(req, rec)
    assert.NoError(t, h.getPosts(c))

    if assert.Equal(t, http.StatusOK, rec.Code) {
        var res postsResponse
        err := json.Unmarshal(rec.Body.Bytes(), &res)
        assert.NoError(t, err)
        assert.Equal(t, 2, len(res.Posts))
    }
}

Run a test

$ cd handler
$ go test -run TestGetPosts
...
ok go-echo-starter/handler 1.380s

Implement update API

Apply auth middleware for authentication to prevent others from updating their posts. What this middleware does is to get the firebase user id from the firebase idToken set in the http header Authorization: Bearer xxxxx, search the users table using the UID as a key, and set the result in The result is set in context.

In the handler, if the user can be retrieved from the context, authentication succeeds; if not, authentication fails.

user := context.Get("user")
if user == nil {
    // authentication fails
} else {
    // authentication succeeded
}

Implementation

handler/post.go

type postResponse struct { { type postResponse struct
    post model.Post `json: "post"`.
}

type updatePostRequest struct { { { type updatePostRequest
    title string `json: "title" validate: "required"`
    body string `json: "body" validate: "required"`
}

func (h *Handler) updatePost(c echo.Context) error { // get user information
    // get user information
    u := c.Get("user")

    if u == nil {
        return c.JSON(http.StatusForbidden, nil)


    user := u.(*model.User)

    // get article
    id, _ := strconv.Atoi(c.Param("id"))
    post, err := h.userStore.FindPostByID(id)

    if err ! = nil { {.
        return c.JSON(http.StatusInternalServerError, nil)
    } else if post == nil {
        return c.JSON(http.StatusNotFound, nil)
    }

    // if it is someone else's post, consider it as unauthorized access
    if post.UserID ! = user.ID { { { if post.UserID ! = user.ID { { { if post.UserID !
        return c.JSON(http.StatusForbidden, nil)
    }

    params := &updatePostRequest{}
    if err := c.Bind(params); err ! = nil { {.
        return c.JSON(http.StatusInternalServerError, nil)
    }

    // Validation
    if err := c.Validate(params); err ! = nil { { if err := c.Validate(params); err !
        return c.JSON(
            http.StatusBadRequest,
            ae.NewValidationError(err, ae.ValidationMessages{
                "Title": {"required": "Please enter a title"},
                "Body": {"required": "Please enter a body"},
            }),
        )
    }

    // Update data
    post.Title = params.
    Post.Body = params.Body

    if err := h.userStore.UpdatePost(post); err ! = nil { .
        return c.JSON(http.StatusInternalServerError, nil)
    }

    return c.JSON(http.StatusOK, postResponse{Post: *post}))
}

Validation can be done using go-playground/validator's (https://github.com/go-playground/validator/blob/master/translations/ja/ja.go) functionality, which allows you to display default multilingual display of error messages. However, this app does not use it, but instead defines a map keyed by field name and validation rule name, and uses display fixed messages.

if err := c.Validate(params); err ! = nil {
    return c.JSON(
        http.StatusBadRequest,
        ae.NewValidationError(err, ae.ValidationMessages{
            "Title": {"required": "required Title."},
            "Body": {"required": "required Body"},
        }),
    )
}

Next, call the handler you created when a PATCH request is made in routes with the path /posts

handler/routes.go

func (h *Handler) Register(api *echo.Group) {
    Auth := middleware.AuthMiddleware(h.authStore, h.userStore)
    ...
    api.PATCH("/posts/:id", h.updatePost, auth)
}

Confirmation of operation

Put the firebase idToken obtained above in the http header and check the operation.

$ go run server.go
...

$ curl -X PATCH -H "Content-Type: application/json" \frz
-H "Authorization: Bearer xxxxxxxxxxxxxx" $ curl
-d '{"title": "NewTitle", "body": "NewBody1"}' \
http://localhost:8000/api/posts/1 | jq

{
  "post": {
    "id": 1,
    "title": "NewTitle",
    "body": "NewBody1",
    }
  }
}

Checking for errors when trying to update someone else's article

$ curl -X PATCH -H "Content-Type: application/json" \
-H "Authorization: Bearer xxxxxxxxxxxxxx" \}
-d '{"title": "NewTitle", "body": "NewBody1"}' \
http://localhost:8000/api/posts/2 -v

...
HTTP/1.1 403 Forbidden
...

Writing Tests

Handler tests that you can update your own articles, but not others'.

Update your own article

handler/post_test.go

func TestUpdatePostSuccess(t *testing.T) {
    setup()

    reqJSON := `{"title":"NewTitle", "body":"NewBody"}`
    authMiddleware := middleware.AuthMiddleware(h.authStore, h.userStore)

    req := httptest.NewRequest(echo.PATCH, "/api/posts/:id", strings.NewReader(reqJSON))
    req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
    req.Header.Set(echo.HeaderAuthorization, "Bearer: ValidToken1")

    rec := httptest.NewRecorder()
    c := e.NewContext(req, rec)
    c.SetPath("/api/posts/:id")
    c.SetParamNames("id")
    c.SetParamValues("1")

    err := authMiddleware(func(c echo.Context) error {
        return h.updatePost(c)
    })(c)
    assert.NoError(t, err)

    if assert.Equal(t, http.StatusOK, rec.Code) {
        var res postResponse
        err := json.Unmarshal(rec.Body.Bytes(), &res)
        assert.NoError(t, err)
        assert.Equal(t, "NewTitle", res.Post.Title)
        assert.Equal(t, "NewBody", res.Post.Body)
    }
}

test returns a fixed user id by idToken for the conversion of idToken to Firebase user id, which is done by the authentication middleware.　Use the mock method prepared in base project.

func (f *fakeAuthClient) VerifyIDToken(context context.Context, token string) (*auth.Token, error) {
    var uid string
    if token == "ValidToken" {
        uid = "ValidUID"
        return &auth.Token{UID: uid}, nil
    } else if token == "ValidToken1" {
        uid = "ValidUID1"
        return &auth.Token{UID: uid}, nil
    } else {
        return nil, errors.New("Invalid Token")
    }
}

Trying to update someone else's article.

handler/post_test.go

func TestUpdatePostForbidden(t *testing.T) {
    setup()

    reqJSON := `{"title":"NewTitle", "body":"NewBody"}`
    authMiddleware := middleware.AuthMiddleware(h.authStore, h.userStore)

    req := httptest.NewRequest(echo.PATCH, "/api/posts/:id", strings.NewReader(reqJSON))
    req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
    req.Header.Set(echo.HeaderAuthorization, "Bearer: ValidToken1")

    rec := httptest.NewRecorder()
    c := e.NewContext(req, rec)
    c.SetPath("/api/posts/:id")
    c.SetParamNames("id")
    c.SetParamValues("2")

    err := authMiddleware(func(c echo.Context) error {
        return h.updatePost(c)
    })(c)
    assert.NoError(t, err)
    assert.Equal(t, http.StatusForbidden, rec.Code)
}

test run

$ go test -run TestUpdatePostSuccess
・・・
ok      go-echo-starter/handler 1.380s

$ go test -run TestUpdatePostForbidden
・・・
ok      go-echo-starter/handler 1.380s

Conclusion

Sample we made this time
https://github.com/nrikiji/go-echo-sample/tree/blog-example