DEV Community

Cover image for URL Shortening Service using Go
Siddhesh Khandagale
Siddhesh Khandagale

Posted on

URL Shortening Service using Go

#go #api #backend #tutorial

Ever wondered how services like Bitly turn long, ugly URLs into short, shareable links?

In this tutorial, I’ll walk you through building a URL Shortener from scratch using Go, Redis and Docker. If you’re a budding developer, this project is a fantastic way to level up your skills and learn practical web development.

What You’ll Learn

  • Setting up a Go project with modules.
  • Building RESTful APIs in Go.
  • Using Redis for quick key-value storage.
  • Organizing Go code for readability and scalability.

Prerequisites / Installation:

  • Docker
  • Docker Desktop ( Install Docker Desktop on your system)

Step 1: Initialize Your Project

Start by creating a directory for your project and initializing a Go module:

mkdir url-shortener
cd url-shortener
go mod init github.com/<username>/url-shortener
Enter fullscreen mode Exit fullscreen mode

This will create a go.mod file to manage dependencies.

Step 2: Set Up the Project Structure

To keep things clean and modular, create the following folder structure:

url-shortener/
├── handlers/         # API logic for handling requests
│   └── handlers.go
├── models/           # Data models
│   └── url.go
├── router/           # Routing setup
│   └── router.go
├── storage/          # Redis interactions
│   └── redis-store.go
├── main.go           # Application entry point
├── Dockerfile        # Docker configuration
├── docker-compose.yml
└── go.mod            # Go module file
Enter fullscreen mode Exit fullscreen mode

Step 3: Install Dependencies

We’ll use the following Go packages:

  • github.com/go-redis/redis/v8 for Redis interactions.
  • github.com/gorilla/mux for routing.

Install them using:

go get github.com/go-redis/redis/v8
go get github.com/gorilla/mux
Enter fullscreen mode Exit fullscreen mode

Step 4: Define Data Models

Create models/url.go to define the data structures for request and response:

package models

type ShortenRequest struct {
 URL string `json:"url"` // URL to be shortened
}

type ShortenResponse struct {
 ShortURL string `json:"short_url"` // Generated short URL
}
Enter fullscreen mode Exit fullscreen mode

The ShortenRequest struct represents the input, and ShortenResponse defines the output format.

Step 5: Configure Redis Storage

In storage/redis-store.go, create a RedisStore struct to handle Redis operations:

package storage

import (
 "context"
 "fmt"
 "hash/fnv"
 "net/url"
 "strings"

 "github.com/go-redis/redis/v8"
)

// Context for Redis operations
var ctx = context.Background()

type RedisStore struct {
 Client *redis.Client
 // client *redis.Client
}

// NewRedisStore initializes a new Redis client and returns a RedisStore
func NewRedisStore() *RedisStore {
 rdb := redis.NewClient(&redis.Options{
  Addr: "localhost:6379",
  DB:   0,
 })

 return &RedisStore{Client: rdb}
}

func NewTestRedisStore() *RedisStore {
 rdb := redis.NewClient(&redis.Options{
  Addr: "localhost:6379",
  DB:   1,
 })

 return &RedisStore{Client: rdb}
}

// SaveURL stores the original URL and its shortened version in Redis
func (s *RedisStore) SaveURL(originalURL string) (string, error) {
 // Check if the URL already exists
 shortURL, err := s.Client.Get(ctx, originalURL).Result()
 if err == redis.Nil {
  // URL not found, generate a new short URL
  shortURL = s.generateShortURL(originalURL)
  err = s.Client.Set(ctx, originalURL, shortURL, 0).Err()
  if err != nil {
   return "", err
  }

  // Store the original URL and the short URL in Redis
  err = s.Client.Set(ctx, shortURL, originalURL, 0).Err()
  if err != nil {
   return "", err
  }

  // Increment the domain count in Redis
  domain, err := s.getDomain(originalURL)
  if err != nil {
   return "", err
  }

  err = s.Client.Incr(ctx, fmt.Sprintf("domain:%s", domain)).Err()
  if err != nil {
   return "", err
  }

 } else if err != nil {
  return "", err
 }

 return shortURL, nil
}

// GetOriginalURL retrieves the original URL from Redis using the short URL
func (s *RedisStore) GetOriginalURL(shortURL string) (string, error) {
 originalURL, err := s.Client.Get(ctx, shortURL).Result()
 if err == redis.Nil {
  return "", fmt.Errorf("URL not found")
 } else if err != nil {
  return "", err
 }

 return originalURL, nil
}

// GetDomainCounts retrieves the counts of shortened URLs per domain from Redis
func (s *RedisStore) GetDomainCounts() (map[string]int, error) {
 keys, err := s.Client.Keys(ctx, "domain:*").Result()
 if err != nil {
  return nil, err
 }

 domainCounts := make(map[string]int)

 for _, key := range keys {
  count, err := s.Client.Get(ctx, key).Int()
  if err != nil {
   return nil, err
  }

  domain := strings.TrimPrefix(key, "domain:")
  domainCounts[domain] = count
 }

 return domainCounts, nil
}

// generateShortURL creates a shortened URL string using a hash function
func (s *RedisStore) generateShortURL(originalURL string) string {
 h := fnv.New32a()
 h.Write([]byte(originalURL))
 return fmt.Sprintf("%x", h.Sum32())
}

// getDomain extracts the domain name from a URL
func (s *RedisStore) getDomain(originalURL string) (string, error) {
 parsedURL, err := url.Parse(originalURL)
 if err != nil {
  return "", err
 }

 return strings.TrimPrefix(parsedURL.Host, "www."), nil
}

func (s *RedisStore) FlushTestDB() {
 s.Client.FlushDB(ctx)
}
Enter fullscreen mode Exit fullscreen mode
  • SaveURL: checks if a URL exists and generates a short URL if it doesn’t.
  • GetOriginalURL: Retrieves the original URL from the short URL.
  • GetDomainCounts: Fetches statistics for the top domains.
  • generateShortURL: hashes the original URL using FNV-32. (fnv.New32a() generates a 32-bit hash for a given URL.)
  • getDomain: Extracts the domain name from a URL.

Why FNV-1a?

  • It’s a fast, non-cryptographic hash function.
  • Generates a short, deterministic hash for a given input.
  • Perfect for URL shortening since we don’t need cryptographic security.
  • Reduces collisions compared to basic modulo-based hashing.

Step 6: Build Handlers

Create handlers/handlers.go to implement API logic:

package handlers

import (
 "encoding/json"
 "net/http"
 "sort"

 "github.com/Siddheshk02/url-shortener/models"
 "github.com/Siddheshk02/url-shortener/storage"
 "github.com/gorilla/mux"
)

// Initialize the store as a RedisStore
var store = storage.NewRedisStore()

func ShortenURL(w http.ResponseWriter, r *http.Request) {
 // Decode the request body into a ShortenRequest struct
 var req models.ShortenRequest
 if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
  http.Error(w, err.Error(), http.StatusBadRequest)
  return
 }

 // Save the URL using the store
 shortURL, err := store.SaveURL(req.URL)
 if err != nil {
  http.Error(w, err.Error(), http.StatusInternalServerError)
 }

 // Respond with the shortened URL
 res := models.ShortenResponse{ShortURL: shortURL}
 json.NewEncoder(w).Encode(res)
}

func RedirectURL(w http.ResponseWriter, r *http.Request) {
 // Get the short URL from the request variables
 vars := mux.Vars(r)
 shortURL := vars["shortURL"]

 // Get the original URL from the store
 originalURL, err := store.GetOriginalURL(shortURL)
 if err != nil {
  http.Error(w, err.Error(), http.StatusNotFound)
  return
 }

 // Redirect to the original URL
 http.Redirect(w, r, originalURL, http.StatusMovedPermanently)
}

func GetTopDomains(w http.ResponseWriter, r *http.Request) {
 // Get the domain counts from the store
 domainCounts, err := store.GetDomainCounts()
 if err != nil {
  http.Error(w, err.Error(), http.StatusInternalServerError)
  return
 }

 // Sort the domains by count
 type kv struct {
  Key   string
  Value int
 }

 var sortedDomains []kv
 for k, v := range domainCounts {
  sortedDomains = append(sortedDomains, kv{k, v})
 }

 sort.Slice(sortedDomains, func(i, j int) bool {
  return sortedDomains[i].Value > sortedDomains[j].Value
 })

 // Prepare the top 3 domains to return
 topDomains := make(map[string]int)
 for i, domain := range sortedDomains {
  if i >= 3 {
   break
  }
  topDomains[domain.Key] = domain.Value
 }

 // Respond with the top 3 domains
 json.NewEncoder(w).Encode(topDomains)
}
Enter fullscreen mode Exit fullscreen mode
  • ShortenURL: Handles URL shortening requests.
  • RedirectURL: Redirects users to the original URL.
  • GetTopDomains: Returns the most frequently shortened domains.

Step 7: Set Up Routes

In router/router.go, define all the API routes:

package router

import (
 "github.com/Siddheshk02/url-shortener/handlers"
 "github.com/gorilla/mux"
)

func SetupRouter() *mux.Router {
 // Initialize a new router
 r := mux.NewRouter()

 // Define route for getting the top domains
 r.HandleFunc("/metrics", handlers.GetTopDomains).Methods("GET")

 // Define route for shortening URLs
 r.HandleFunc("/shorten", handlers.ShortenURL).Methods("POST")

 // Define route for redirecting to the original URL
 r.HandleFunc("/{shortURL}", handlers.RedirectURL).Methods("GET")

 return r
}
Enter fullscreen mode Exit fullscreen mode
  • /shorten (POST): Shortens a given URL.
  • /{shortURL} (GET): Redirects to the original URL using the short URL.
  • /metrics (GET): Returns the top 3 most-used domains.

Step 8: Main Application Entry Point

In main.go, initialize the router and start the HTTP server:

package main

import (
 "log"
 "net/http"

 "github.com/Siddheshk02/url-shortener/router"
 "github.com/gorilla/mux"
)

func main() {
 // Setup the router from the router package
 r := router.SetupRouter()

 // Log all registered routes
 r.Walk(func(route *mux.Route, router *mux.Router, ancestors []*mux.Route) error {
  path, err := route.GetPathTemplate()
  if err != nil {
   return err
  }
  log.Println("Registered route:", path)
  return nil
 })

 log.Println("Starting server on 8080")
 log.Fatal(http.ListenAndServe(":8080", r))
}
Enter fullscreen mode Exit fullscreen mode
  • router.SetupRouter: Initializes all routes.
  • Route Logging: Lists all registered routes for debugging.
  • HTTP Server: Runs the app on port 8080.

Step 9: Dockerize the Application

Create a Dockerfile to containerize the app:

# Start with the official Golang image
FROM golang:1.22-alpine

# Set the working directory inside the container
WORKDIR /app

# Copy the Go modules files first and download dependencies
COPY go.mod go.sum ./
RUN go mod download

# Copy the rest of the application code
COPY . .

# Build the application
RUN go build -o url-shortener

# Expose the port on which the app will run
EXPOSE 8080

# Command to run the executable
CMD ["./url-shortener"]
Enter fullscreen mode Exit fullscreen mode

The Dockerfile builds the Go application into a Docker image and sets it up to run on port 8080.

Step 10: Create a docker-compose.yml File

Use docker-compose.yml to define both the app and the Redis service:

version: '3'

services:
  url-shortener:
    image: siddheshk02/url-shortener:latest
    ports:
      - "8080:8080"
    depends_on:
      - redis
    environment:
      - REDIS_HOST=redis
    networks:
      - app-network

  redis:
    image: redis
    ports:
      - "6379:6379"
    networks:
      - app-network

networks:
  app-network:
    driver: bridge
Enter fullscreen mode Exit fullscreen mode
  • url-shortener Service: Runs the Go app and depends on Redis.
  • redis Service: Provides the Redis database for storage.
  • network: Ensures communication between the two services.

Step 11: Run the Application

Build and run the application using Docker Compose:

docker-compose up --build
Enter fullscreen mode Exit fullscreen mode

Output:

The Go app runs on http://localhost:8080.
Redis is available on port 6379.

Step 12: Test the API Endpoints

Use Postman, cURL, or a browser to test the service:

  • Shorten a URL: 
curl -X POST -H "Content-Type: application/json" \
-d '{"url": "https://www.example.com"}' \
http://localhost:8080/shorten
Enter fullscreen mode Exit fullscreen mode

Response:

{
    "short_url": "5c285a56"
}
Enter fullscreen mode Exit fullscreen mode 
curl http://localhost:8080/metrics
Enter fullscreen mode Exit fullscreen mode

Response :

{
    "example.com": 5,
    "github.com": 3,
    "google.com": 2
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

Congratulations! 🎉 You’ve built a fully functional URL Shortening Service in Go. This project covered:

  • URL Shortening: Generating short URLs.
  • Redirection: Navigating users from short to original URLs.
  • Domain Tracking: Analyzing the most-used domains.
  • Dockerization: Deploying the app in a containerized environment.

You can get the complete code repository here.

Next Steps
Here are some ways to enhance the project:

  • Custom Short URLs: Allow users to create custom short links.
  • Analytics: Track the number of visits per short URL.
  • Expiration: Add expiration times for short URLs.

That’s it for this Tutorial. If this blog was helpful to you do share it with other and to get more information about Golang concepts, projects, etc. and to stay updated on the Tutorials do follow Siddhesh on Twitter and GitHub.

Until then Keep Learning, Keep Building 🚀🚀

Top comments (0)

Read next

shayy profile image

Run DeepSeek-R1 on Your Laptop with Ollama

Shayan -

anisharaz profile image

Dev.to on your own domain xyz.com.

ANISH ARAZ -

sadanand_dodawadakar_10b2 profile image

Hidden Power of Go: Unveiling the Secrets of a Robust Language

Sadanand Dodawadakar -

bkthemes profile image

10 Design Fails to Learn From

Brian Keary -