Singleflight: Reduce Costs and Improve Performance in Go Applications

As software engineers, we should always consider how to improve the performance of our applications while also reducing costs. Achieving both of these goals simultaneously is not always easy; often, we are forced to choose between being cheaper or faster.

In this article, I want to show you how we can achieve both goals by using singleflight.

What is Singleflight?

Singleflight is a standard Go library (golang.org/x/sync/singleflight) that prevents redundant calls to the same function. It achieves this by grouping concurrent requests to the same resource and returning the first result to all subsequent requests.

Use cases examples

• Queries to external APIs charged per request.
• Read operations in databases.
• Prevention of rate limiting in APIs with request limits.

Example: Product Price Search System

• An e-commerce platform where users search for product prices.
• The price is obtained from an external API that charges per request.
• Multiple users can search for the price of the same product simultaneously.

Code without singleflight

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "time"
)

var cost float64


func fetchProductPrice(productID string) (float64, error) {
    log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID)
    time.Sleep(2 * time.Second) // Simulates latency
    cost += 0.01
    return 99.99, nil 
}


func getProductPriceHandler(w http.ResponseWriter, r *http.Request) {
    productID := r.URL.Query().Get("id")

    price, err := fetchProductPrice(productID)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    response := map[string]interface{}{
        "product_id": productID,
        "price":      price,
    }


    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func getCost(w http.ResponseWriter, r *http.Request) {

    response := map[string]interface{}{
        "total_cost": fmt.Sprintf("%.2f", cost),
    }


    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func clearCosts(w http.ResponseWriter, r *http.Request) {
    cost = 0
}

func main() {

    mux := http.NewServeMux()

    mux.HandleFunc("/products/{id}/price", getProductPriceHandler)
    mux.HandleFunc("/costs", getCost)
    mux.HandleFunc("/clear-costs", clearCosts)


    log.Println("API running without singleflight on port :8080...")
    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatalf("Could not start server: %s\n", err.Error())
    }
}

Problem: If 100 users search for the price of the same product at the same time, the system will make 100 calls to the external API, generating a cost of $1.00 and increasing latency.

Code with singleflight

package main

import (
    "encoding/json"
    "log"
    "net/http"
    "time"

    "golang.org/x/sync/singleflight"
)

var (
    group singleflight.Group
    cost  float64
)


func fetchProductPrice(productID string) (float64, error) {
    log.Printf("[COST: $0.01] Calling external API for product: %s\n", productID)
    time.Sleep(2 * time.Second) 
    cost += 0.01
    return 99.99, nil 
}

func getProductPriceHandler(w http.ResponseWriter, r *http.Request) {
    productID := r.URL.Query().Get("id")

    // Uses singleflight to avoid redundant calls
    result, err, _ := group.Do(productID, func() (interface{}, error) {
        return fetchProductPrice(productID)
    })
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }


    price := result.(float64)


    response := map[string]interface{}{
        "product_id": productID,
        "price":      price,
    }


    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func getCost(w http.ResponseWriter, r *http.Request) {

    response := map[string]interface{}{
        "total_cost": fmt.Sprintf("%.2f", cost),
    }
    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(response); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
}

func clearCosts(w http.ResponseWriter, r *http.Request) {
    cost = 0
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/products/{id}/price", getProductPriceHandler)
    mux.HandleFunc("/costs", getCost)
    mux.HandleFunc("/clear-costs", clearCosts)

    log.Println("API running with singleflight on port :8081...")
    if err := http.ListenAndServe(":8081", mux); err != nil {
        log.Fatalf("Could not start server: %s\n", err.Error())
    }
}

Improvement: Only one call is made to the external API, regardless of how many users are searching for the price of the same product. This reduces the cost to $0.01 and decreases latency.

Tests using Vegeta

Without Singleflight

vegeta attack -duration=2s -rate=200 -workers=200 -targets=targets_without_singleflight.txt | vegeta report

Requests      [total, rate, throughput]         400, 200.47, 100.08
Duration      [total, attack, wait]             3.997s, 1.995s, 2.001s
Latencies     [min, mean, 50, 90, 95, 99, max]  2.001s, 2.002s, 2.002s, 2.002s, 2.002s, 2.002s, 2.003s
Bytes In      [total, mean]                     12800, 32.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:400  
Error Set:

1. Latency:
   • All requests took ~2 seconds to complete.
   • This happens because each request made an independent call to the external API, which has a simulated latency of 2 seconds.
2. Throughput:
   • The throughput was 100.08 requests per second.
   • This indicates that the system was able to process about 100 requests per second, but with high latency.

3. Cost:

   • Since each request made a call to the external API, the cost would be 4.00(400 requests × 0.01 per call).

   • GET http://localhost:8080/costs
     

     {"total_cost":"4.00"}
     

With singleflight

vegeta attack -duration=2s -rate=200 -workers=200 -targets=targets_with_singleflight.txt | vegeta report  
Requests      [total, rate, throughput]         400, 200.46, 198.32
Duration      [total, attack, wait]             2.017s, 1.995s, 21.452ms
Latencies     [min, mean, 50, 90, 95, 99, max]  15.337ms, 1.014s, 1.019s, 1.811s, 1.911s, 1.991s, 2.009s
Bytes In      [total, mean]                     12800, 32.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:400  
Error Set:

1. Latency:
   • The minimum latency was 15.337ms, and the average was 1.014s.
   • This happens because only one call was made to the external API (taking ~2 seconds), and the other requests received the result in less than 22ms.
2. Throughput:
   • The throughput increased to 198.32 requests per second.
   • This indicates that the system was able to process almost double the number of requests per second, thanks to the reduced latency for most requests.

3. Cost:

   • Since only one call was made to the external API, the cost would be 0.01 (1 request × 0.01 per call).

   • GET http://localhost:8081/costs
     

     {"total_cost":"0.01"}
     

Direct Comparison

Metric	Without singleflight	With singleflight	Gain
Average Latency	2.002s	1.014s	~49%
Throughput	100.08 req/s	198.32 req/s	~98%
Cost	$4.00	$0.01	99.75%

---

1. Latency Reduction:
   • Singleflight reduced the average latency from 2.002s to 1.014s, a gain of ~49%.
   • This happens because most requests were resolved in less than 22ms, while only the first request took ~2 seconds.
2. Throughput Increase:
   • The throughput increased from 100.08 req/s to 198.32 req/s, a gain of ~98%.
   • This indicates that the system was able to process almost double the number of requests per second, thanks to the reduced latency.
3. Cost Reduction:
   • The cost dropped from 4.00 to 0.01, a savings of 99.75%.
   • This happens because only one call was made to the external API, instead of 400.

---

When to Use (and When Not to Use)

• Use:
  • For read operations (queries to APIs or databases).
  • When the resource is idempotent (does not change the system state).
• Do not use:
  • For write operations (creation, update, deletion).
  • When the result may vary between calls.

Conclusion

Singleflight is a powerful tool to optimize Go applications, reducing costs and improving performance in high-concurrency scenarios. Try implementing it in your projects and see the difference!

Try it out: Add singleflight to your next project and share the results!! 🚀

check codes here https://github.com/guil95/singleflight