Use Anthropic Claude 3 models to build generative AI applications with Go

Anthropic's Claude 3 is a family of AI models with different capabilities and cost for a variety of tasks:

• Claude 3 Haiku is a compact and fast model that provides near-instant responsiveness
• Claude 3 Sonnet provides a balance between skills and speed
• Claude 3 Opus is for highly complex tasks when you that need high intelligence and performance

Claude 3 models are multi-modal. This means that they can accept both text and images as inputs (although they can only output text). Let's learn how to use the Claude 3 models on Amazon Bedrock with Go.

Basic examples

Refer to Before You Begin section in this blog post to complete the prerequisites for running the examples. This includes installing Go, configuring Amazon Bedrock access and providing necessary IAM permissions.

Amazon Bedrock abstracts multiple models via a uniform set of APIs that exchange JSON payloads. Same applies for Claude 3.

Let's start off with a simple example using AWS SDK for Go (v2).

You can run it as such:

git clone https://github.com/abhirockzz/claude3-bedrock-go
cd claude3-bedrock-go

go run basic/main.go

The response may (or may not) be slightly different in your case:

request payload:
 {"anthropic_version":"bedrock-2023-05-31","max_tokens":1024,"messages":[{"role":"user","content":[{"type":"text","text":"Hello, what's your name?"}]}]}
response payload:
 {"id":"msg_015e3SJ99WF6p1yhGTkc4HbK","type":"message","role":"assistant","content":[{"type":"text","text":"My name is Claude. It's nice to meet you!"}],"model":"claude-3-sonnet-28k-20240229","stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":14,"output_tokens":15}}
response string:
 My name is Claude. It's nice to meet you!

You can refer to the complete code here. I will break it down to make this simpler for you:

We start by creating the JSON payload - it's modeled as a struct (Claude3Request):

    msg := "Hello, what's your name?"

    payload := Claude3Request{
        AnthropicVersion: "bedrock-2023-05-31",
        MaxTokens:        1024,
        Messages: []Message{
            {
                Role: "user",
                Content: []Content{
                    {
                        Type: "text",
                        Text: msg,
                    },
                },
            },
        },
    }

  payloadBytes, err := json.Marshal(payload)

InvokeModel is used to call the model. The JSON response is converted to a struct (Claude3Response) and the text response is extracted from it.

//....
  output, err := brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
      Body:        payloadBytes,
      ModelId:     aws.String(modelID),
      ContentType: aws.String("application/json"),
  })

  var resp Claude3Response
  err = json.Unmarshal(output.Body, &resp)

  log.Println("response string:\n", resp.ResponseContent[0].Text)

Chat with streaming

Now moving on to a common example which involves a conversational exchange. We will also add a streaming element to for better experience - the client application does not need to wait for the complete response to be generated for it start showing up in the conversation.

You can run it as such:

go run chat-streaming/main.go

You can refer to the complete code here.

A streaming based implementation is a bit more involved. First off, we use InvokeModelWithResponseStream (instead of Invoke).

    output, err := brc.InvokeModelWithResponseStream(context.Background(), &bedrockruntime.InvokeModelWithResponseStreamInput{
        Body:        payloadBytes,
        ModelId:     aws.String(modelID),
        ContentType: aws.String("application/json"),
    })

    resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
        fmt.Print(string(part))
        return nil
    })

To process it's output, we use:

    resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
        fmt.Print(string(part))
        return nil
    })
//...

Here are a few bits from the processStreamingOutput function - you can check the code here. The important thing to understand is how the partial responses are collected together to produce the final output.

func processStreamingOutput(output *bedrockruntime.InvokeModelWithResponseStreamOutput, handler StreamingOutputHandler) (Claude3Response, error) {

  //...
    for event := range output.GetStream().Events() {
        switch v := event.(type) {
        case *types.ResponseStreamMemberChunk:

            var pr PartialResponse
            err := json.NewDecoder(bytes.NewReader(v.Value.Bytes)).Decode(&pr)
            if err != nil {
                return resp, err
            }

            if pr.Type == partialResponseTypeContentBlockDelta {
                handler(context.Background(), []byte(pr.Delta.Text))
                combinedResult += pr.Delta.Text
            } else if pr.Type == partialResponseTypeMessageStart {
                resp.ID = pr.Message.ID
                resp.Usage.InputTokens = pr.Message.Usage.InputTokens
            } else if pr.Type == partialResponseTypeMessageDelta {
                resp.StopReason = pr.Delta.StopReason
                resp.Usage.OutputTokens = pr.Message.Usage.OutputTokens
            }
  //...
    }

  resp.ResponseContent[0].Text = combinedResult

    return resp, nil
}

Use Claude Haiku with images

All the Claude3 models can accept images as inputs. Haiku is deemed to be good at OCR (optical), understanding images, etc. Let's make use of Claude Haiku the upcoming examples - no major changes required, except for the model ID.

The example uses this image (actual StackOverflow question) along with this prompt Transcribe the code in the question. Only output the code.

You can run it as such:

go run images/main.go

This is the response that Claude Haiku came up with. Impressive!

 func test() {
    cluster := gocql.NewCluster("hostIP")
    cluster.ProtoVersion = 4
    cluster.Authenticator = gocql.PasswordAuthenticator{
        Username: "<username>",
        Password: "<password>",
    }
    cluster.Keyspace = "myspace"
    cluster.Consistency = gocql.One
    session, err := cluster.CreateSession()
    if err != nil {
        panic(err)
    }
    defer session.Close()
}

I tried another one: "Can you suggest a solution to the question?"

And here is the response:

 Based on the error messages shown in the image, it seems that the issue is related to connecting to the Cassandra database using the gocql driver (Go: 1.8.3). The errors indicate that the application is unable to dial the control connection, create a session, or connect to the initial hosts within the timeout period.

A few things you could check to troubleshoot this issue:

1. Verify the connection details (host, port, username, password) are correct and match the Cassandra cluster configuration.

2. Ensure the Cassandra cluster is up and running and accessible from the application host.

3. Check the firewall settings to make sure the application host is able to connect to the Cassandra cluster on the required ports.

4. Inspect the Cassandra server logs for any errors or clues about the connection issue.

5. Try increasing the timeout values in the gocql configuration, as the current timeouts may be too short for the Cassandra cluster to respond.

6. Ensure the gocql version (1.8.3) is compatible with the Cassandra version (3.10.0) you are using.

7. Consider adding some error handling and retry logic in your code to gracefully handle connection failures and attempt to reconnect.

Without more context about your specific setup and environment, it's difficult to pinpoint the exact issue. However, the steps above should help you investigate the problem and find a solution.

You can refer to the complete code here.

Here is what the JSON payload for the request:

    payload := Claude3Request{
        AnthropicVersion: "bedrock-2023-05-31",
        MaxTokens:        1024,
        Messages: []Message{
            {
                Role: "user",
                Content: []Content{
                    {
                        Type: "image",
                        Source: &Source{
                            Type:      "base64",
                            MediaType: "image/jpeg",
                            Data:      imageContents,
                        },
                    },
                    {
                        Type:   "text",
                        Text:   msg,
                        Source: nil,
                    },
                },
            },
        },
    }

The imageContents in Data attribute is the base64 encoded version of the image which is calculated like this:

func readImageAsBase64(filePath string) (string, error) {

    imageFile, err := os.ReadFile(filePath)

    if err != nil {
        return "", err
    }

    encodedString := base64.StdEncoding.EncodeToString(imageFile)
    return encodedString, nil
}

You can try a different image (for example this one), and check if it's able to calculate the cost of all menu items.

Combine text and image conversations

Image and text data are not exclusive. The message structure (JSON payload) is flexible enough to support both.

You can refer to the complete code here.

Here is an example in which you can mix and match text and image based messages. You can run it as such:

go run multi-modal-chat-streaming/main.go

Here is the response (I used the same StackOverflow image):

Wrap up

You don't have to depend on Python to build generative AI applications. Thanks to AWS SDK support, you can use the programming language of your choice (including Go!) to integrate with Amazon Bedrock.