Go and MongoDB: Building a CRUD API from Scratch

Want to create a dynamic web application with a robust backend? Look no further than Go and MongoDB! This powerful combination allows you to build scalable, efficient APIs that handle data creation, reading, updating, and deletion (CRUD) with ease.

In this beginner-friendly guide, we'll walk through the process of building a simple CRUD API using Go and MongoDB. We'll cover the essential steps, provide code examples, and sprinkle in useful tips along the way.

Getting Started

First things first, let's set up our environment:

1. Go Installation: Download and install the latest version of Go from https://go.dev/dl/.
2. MongoDB Setup: If you don't have MongoDB running, you can download and install it from https://www.mongodb.com/try/download/community.
3. IDE or Text Editor: Choose your preferred coding environment. Some popular options include VS Code, GoLand, or Atom.

Project Structure:

Create a new project directory and organize your files like this:

my-crud-api/
├── main.go
├── models/
│   └── user.go
├── handlers/
│   └── user.go
└── config/
    └── config.go

Defining Our Model

Let's start with defining our data model. For this example, we'll create a simple User struct:

// models/user.go
package models

import (
    "go.mongodb.org/mongo-driver/bson/primitive"
)

type User struct {
    ID     primitive.ObjectID `bson:"_id,omitempty"`
    Name   string             `bson:"name,omitempty"`
    Email  string             `bson:"email,omitempty"`
    Age    int                `bson:"age,omitempty"`
    Active bool               `bson:"active,omitempty"`
}

Explanation:

• We use primitive.ObjectID from the mongo-driver package to represent the unique MongoDB document ID.
• The bson tags are crucial for mapping our Go struct fields to the corresponding fields in our MongoDB documents.

Connecting to MongoDB

We need to establish a connection to our MongoDB database. Create a config.go file in the config directory:

// config/config.go
package config

import (
    "context"
    "fmt"
    "os"

    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

func ConnectToMongoDB() (*mongo.Client, error) {
    uri := os.Getenv("MONGODB_URI")
    if uri == "" {
        return nil, fmt.Errorf("MONGODB_URI is not set")
    }

    clientOptions := options.Client().ApplyURI(uri)
    client, err := mongo.Connect(context.Background(), clientOptions)
    if err != nil {
        return nil, err
    }

    err = client.Ping(context.Background(), nil)
    if err != nil {
        return nil, err
    }

    return client, nil
}

Explanation:

• We use os.Getenv to retrieve the MongoDB connection URI from the environment variable MONGODB_URI. Make sure to set this variable in your environment.
• We use the mongo-driver package to connect to the MongoDB database and perform basic operations like pinging the database.

Building Handlers

Now, let's build the API handlers for our CRUD operations. In the handlers directory, create a user.go file:

// handlers/user.go
package handlers

import (
    "context"
    "encoding/json"
    "fmt"
    "net/http"

    "github.com/your-username/my-crud-api/config"
    "github.com/your-username/my-crud-api/models"
    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
)

// Create a new user
func CreateUser(w http.ResponseWriter, r *http.Request) {
    client, err := config.ConnectToMongoDB()
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer client.Disconnect(context.Background())

    var user models.User
    if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    collection := client.Database("your_database_name").Collection("users")
    result, err := collection.InsertOne(context.Background(), user)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(result)
}

// Get all users
func GetAllUsers(w http.ResponseWriter, r *http.Request) {
    client, err := config.ConnectToMongoDB()
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer client.Disconnect(context.Background())

    collection := client.Database("your_database_name").Collection("users")
    cursor, err := collection.Find(context.Background(), bson.D{})
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer cursor.Close(context.Background())

    var users []models.User
    for cursor.Next(context.Background()) {
        var user models.User
        if err := cursor.Decode(&user); err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        users = append(users, user)
    }

    json.NewEncoder(w).Encode(users)
}

// Get a user by ID
func GetUserByID(w http.ResponseWriter, r *http.Request) {
    client, err := config.ConnectToMongoDB()
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer client.Disconnect(context.Background())

    id, err := primitive.ObjectIDFromHex(r.URL.Query().Get("id"))
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    collection := client.Database("your_database_name").Collection("users")
    var user models.User
    if err := collection.FindOne(context.Background(), bson.M{"_id": id}).Decode(&user); err != nil {
        http.Error(w, err.Error(), http.StatusNotFound)
        return
    }

    json.NewEncoder(w).Encode(user)
}

// Update a user
func UpdateUser(w http.ResponseWriter, r *http.Request) {
    client, err := config.ConnectToMongoDB()
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer client.Disconnect(context.Background())

    id, err := primitive.ObjectIDFromHex(r.URL.Query().Get("id"))
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    var updatedUser models.User
    if err := json.NewDecoder(r.Body).Decode(&updatedUser); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    collection := client.Database("your_database_name").Collection("users")
    filter := bson.M{"_id": id}
    update := bson.M{"$set": updatedUser}
    result, err := collection.UpdateOne(context.Background(), filter, update)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(result)
}

// Delete a user
func DeleteUser(w http.ResponseWriter, r *http.Request) {
    client, err := config.ConnectToMongoDB()
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    defer client.Disconnect(context.Background())

    id, err := primitive.ObjectIDFromHex(r.URL.Query().Get("id"))
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    collection := client.Database("your_database_name").Collection("users")
    result, err := collection.DeleteOne(context.Background(), bson.M{"_id": id})
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(result)
}

Explanation:

• We implement the CRUD operations: CreateUser, GetAllUsers, GetUserByID, UpdateUser, and DeleteUser.
• Each function connects to MongoDB, retrieves the collection, performs the respective operation, and returns a JSON response.
• We handle potential errors and return appropriate HTTP status codes.

Setting up the Main Application

Finally, let's tie everything together in our main.go file:

// main.go
package main

import (
    "fmt"
    "log"
    "net/http"

    "github.com/your-username/my-crud-api/handlers"
)

func main() {
    http.HandleFunc("/users", handlers.CreateUser)
    http.HandleFunc("/users", handlers.GetAllUsers)
    http.HandleFunc("/users/", handlers.GetUserByID)
    http.HandleFunc("/users/", handlers.UpdateUser)
    http.HandleFunc("/users/", handlers.DeleteUser)

    fmt.Println("Server running on port 8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Explanation:

• We register the API handlers with the corresponding HTTP endpoints.
• We start the server and listen on port 8080.

Running the API

1. Environment Variable: Set the MONGODB_URI environment variable with your MongoDB connection string.
2. Build and Run: Build the Go application using go build and then run it using ./my-crud-api.

Testing the API

You can test your API using tools like Postman or curl.

• Create: Send a POST request to /users with a JSON payload containing user details.
• Read: Send a GET request to /users to retrieve all users or to /users/?id={user_id} to get a specific user.
• Update: Send a PUT request to /users/?id={user_id} with a JSON payload containing updated user details.
• Delete: Send a DELETE request to /users/?id={user_id} to delete a user.

Tips for Success

• Error Handling: Always handle potential errors and return meaningful HTTP status codes.
• Security: Implement proper authentication and authorization mechanisms for your API.
• Database Design: Design your database schema thoughtfully to optimize performance and scalability.
• Documentation: Document your API endpoints, request/response formats, and error codes.

Congratulations! You've successfully built a basic CRUD API using Go and MongoDB. With this foundation, you can expand your API to handle more complex functionalities and build impressive web applications. Keep learning and exploring the endless possibilities of Go and MongoDB!

Comments

[Dhiren Pathak]

Nice

[Martin Baun]

thanks for explaining this in detail, Mohd, I especially appreciated the little tidbits for success :)

[Nikhil Upadhyay]

Nice tutorial

[Adrian Goodyer]

Great article Mohd, congrats!

Which tool would yourself (or anyone else) recommend to manage migrations with Go and MongoDB?

Atlas and Goose are my preference, but are SQL focused so I use go-migrations - but the migrations are written in JSON and I'd really like to write the migrations in Go instead!

[Mohd Aquib]

To manage MongoDB migrations in Go without using JSON, use the migrate library with the MongoDB driver and write migrations programmatically in Go. Here’s a concise guide:

Install dependencies:

   go get -u -d github.com/golang-migrate/migrate/cmd/migrate
   go get -u github.com/golang-migrate/migrate/v4/database/mongodb
   go get -u github.com/golang-migrate/migrate/v4/source/file

Create a migration in Go:

   package main

   import (
       "github.com/golang-migrate/migrate/v4"
       "github.com/golang-migrate/migrate/v4/database/mongodb"
       "github.com/golang-migrate/migrate/v4/source/file"
       "log"
   )

   func main() {
       mongoURL := "mongodb://localhost:27017/mydatabase"

       source, err := (&file.File{}).Open("file://path/to/migrations")
       if err != nil {
           log.Fatal(err)
       }

       database, err := mongodb.WithInstance(mongoURL, &mongodb.Config{})
       if err != nil {
           log.Fatal(err)
       }

       m, err := migrate.NewWithInstance(
           "file",
           source,
           "mongodb",
           database,
       )
       if err != nil {
           log.Fatal(err)
       }

       if err := m.Up(); err != nil && err != migrate.ErrNoChange {
           log.Fatal(err)
       }

       log.Println("Migrations applied successfully!")
   }

Write migrations as Go scripts.

This setup allows you to handle MongoDB migrations using Go code.

[Emmanuel Soetan]

Ever since I learned about goose I have used goose for migrations for PostgreSQL. I feel it gives me more control.

[RoyLizhiyu]

how is authentication and authorization handled in the example?

[Danang Firmanto]

good