Building a Scalable Newsletter System with Go: From Test to Production

In today’s digital world, email newsletters are an essential tool for staying in touch with your audience. Whether it’s product updates, company news, or marketing campaigns, a reliable newsletter system can make a big difference in how you connect with your customers.

In this article, I’ll show you how we built a scalable newsletter system using Go and GoFr that integrates seamlessly with Zoho CRM. I’ll walk you through the architecture, implementation, and best practices we followed to take it from testing to full production.

The Challenge

Our team faced a common but tricky challenge: sending personalized newsletters to thousands of leads while ensuring:

1. Reliability – No dropped emails.
2. Rate Limiting – Respecting API constraints.
3. Safe Testing – Testing thoroughly before going live.
4. Maintainability – Making it easy for future developers to work on.

We needed a solution that could handle all of this efficiently.

System Architecture

We broke the system into three main components:

1. Newsletter Generator – Handles the creation of HTML newsletters from templates.
2. Zoho CRM Integration – Manages lead data and sends emails.
3. Web API – Provides endpoints for triggering newsletter actions.

Let’s dive into each component and see how it works.

1. Newsletter Generator
The first step was creating a flexible way to generate newsletters. We used Go’s text/template package to build HTML templates that could be customized dynamically based on input data.

Here’s how it works:

func (s *NewsletterService) GenerateNewsletter(req models.NewsletterRequest) (string, error) {
    tmpl := template.New("newsletter").Funcs(template.FuncMap{
        "add": func(a, b int) int { return a + b },
        "title": strings.Title,
    })

    tmpl, err := tmpl.Parse(s.templateHtml)
    if err != nil {
        return "", fmt.Errorf("failed to parse template: %w", err)
    }

    data := struct {
        Version      string
        TextSections map[string][]models.TextContent
        Blogs        []models.LinkContent
    }{
        Version:      req.Version,
        TextSections: make(map[string][]models.TextContent),
    }

    for sectionType, content := range req.Headings {
        switch sectionType {
        case "heading":
            var headingContent models.TextContent
            if err := json.Unmarshal(content, &headingContent); err != nil {
                return "", fmt.Errorf("failed to parse heading: %w", err)
            }
            data.TextSections["heading"] = []models.TextContent{headingContent}
        case "highlights", "features", "updates":
            var items []models.TextContent
            if err := json.Unmarshal(content, &items); err != nil {
                return "", fmt.Errorf("failed to parse %s: %w", sectionType, err)
            }
            data.TextSections[sectionType] = items
        case "blogs":
            if err := json.Unmarshal(content, &data.Blogs); err != nil {
                return "", fmt.Errorf("failed to parse blogs: %w", err)
            }
        }
    }

    var buf bytes.Buffer
    if err := tmpl.Execute(&buf, data); err != nil {
        return "", fmt.Errorf("failed to execute template: %w", err)
    }

    return buf.String(), nil
}

This approach allows us to easily add or modify sections in the newsletter without changing the code structure.

2. Zoho CRM Integration
The core of our system is Zoho CRM integration. It handles everything from authenticating with Zoho’s API to retrieving leads and sending emails.

Here’s an overview of the email-sending process:

1. Retrieve leads from Zoho CRM (either all leads or just test leads).
2. Send emails in batches to avoid hitting API rate limits.
3. Implement retry logic for failed attempts.

Here’s a snippet of our email-sending logic:

func (s *ZohoCRMService) SendNewsletterToLeads(ctx *gofr.Context, newsletter models.EmailContent, testMode bool) (models.EmailResponse, error) {
    var leads []models.Lead
    var err error

    if testMode && s.config.SafetyFlag != "enabled" {
        return models.EmailResponse{
            Success: false,
            Message: "Test mode is not enabled. Set PRODUCTION_TEST_MODE=enabled to enable test mode.",
        }, errors.New("test mode is not enabled")
    }

    if testMode {
        leads, err = s.GetTestLeads(ctx)
    } else {
        leads, err = s.GetAllLeads(ctx)
    }
    if err != nil || len(leads) == 0 {
        return models.EmailResponse{
            Success: false,
            Message: "No leads found",
        }, errors.New("no leads found")
    }

    successCount := 0
    batchSize := s.config.BatchSize

    for i := 0; i < len(leads); i += batchSize {
        endIdx := i + batchSize
        if endIdx > len(leads) {
            endIdx = len(leads)
        }

        batchLeads := leads[i:endIdx]

        for _, lead := range batchLeads {
            for retry := 0; retry < s.config.MaxRetries; retry++ {
                result, sendErr := s.SendEmail(ctx, lead, newsletter.Subject, newsletter.Content, testMode)
                if sendErr == nil && result {
                    successCount++
                    break
                }
                time.Sleep(time.Duration(s.config.RetryDelay) * time.Second)
            }
            time.Sleep(time.Second)
        }

        time.Sleep(time.Duration(s.config.BatchDelay) * time.Second)
    }

    return models.EmailResponse{
        Success: successCount > 0,
        Message: fmt.Sprintf("Successfully sent newsletter to %d out of %d leads", successCount, len(leads)),
        Sent:    successCount,
        Total:   len(leads),
    }, nil
}

3. Web API
Finally, we built a Web API that allows users to interact with the system via HTTP endpoints. For example:

1. Create newsletters (/newsletter)
2. Send newsletters (/newsletter/send)
3. Get a summary of leads (/newsletter/leads)

Here’s an example handler for sending newsletters:

func (h *ZohoCRMHandler) SendNewsletter(ctx *gofr.Context) (any, error) {
    var req models.EmailRequest

    if err := ctx.Bind(&req); err != nil {
       return models.EmailResponse{
          Success: false,
          Message: "Invalid request body: " + err.Error(),
       }, err
    }

    resp, err := h.service.SendNewsletterToLeads(ctx, req.Newsletter, req.TestMode)

    resp.ElapsedTime = time.Since(startTime).String()

    if err != nil {
       ctx.Logger.Error("Error sending newsletter: " + err.Error())
       return resp, nil
    }

    return resp, nil
}

Key Features for Production Readiness

To ensure our system was ready for production use, we added several important features:

1. Environment-Based Configuration – All parameters (e.g., batch size and retry limits) are configurable via environment variables.
2. Test Mode with Safety Flags – Prevent accidental emails by requiring explicit enablement of test mode.
3. Rate Limiting and Batch Processing – Respect API limits by processing emails in batches.
4. Retry Logic – Handle transient failures gracefully by retrying failed attempts.
5. Comprehensive Logging – Add detailed logs for troubleshooting issues.

Lessons Learned

1. Building this system taught us some valuable lessons:
2. Always start with a safe testing mechanism.
3. Respect API rate limits by batching requests.
4. Plan for failures by implementing retries.
5. Use logging extensively—it’s your best friend during debugging.
6. Make everything configurable so the system can adapt without code changes.

Conclusion

A well-designed newsletter system can transform how you engage with your audience. By focusing on scalability and reliability from the start, we built a solution that can handle both small-scale tests and large-scale production deployments.

If you’re building something similar or have questions about any part of this process—let me know! I’d love to help.
also check out GoFr which made this integration seamless