Ollama 0.5 Is Here: Generate Structured Outputs

Introduction

Whats up everyone! This is Dev. Yesterday, that is, 6th December, Ollama released 0.5 version which supports Structured Outputs. This is really exciting and problem solving.

This blog contains information on how to define the required structure and a tutorial on how to get the output. In this tutorial, I am going to use Ollama's Docker image.

Tutorial.

Step 1: docker-compose.yml

Firstly, defining the requirements for pulling the Ollama's docker image. Create a docker-compose.yml file with the following code.

services:
  ollama:
    image: ollama/ollama:latest
    restart: always
    ports:
      - "8080:11434"

Note: If you previously pulled Ollama's image, make sure to remove the previous image using docker rmi ollama/ollama, so that it pulls the latest image when you run the docker compose file.

Finally, start the container using, docker compose up --build -d.

Step 2: Pull the model

Once the container is started, now its time to pull the required model. For that purpose, we need to get into the container using docker exec -it <Container's Name> bash. Replace the 'Container's Name' with the name of the container. This starts an interactive bash shell terminal.

From this shell, to get the required model, use the command, ollama pull <Model Name>. Replace the 'Model Name' with the name of the required model. For the tutotial, I am using llama3.2:latest.

Once the model is pulled, you can exit the shell using exit command.

Step 3: Prompt the model

Now its time to prompt the model. You can use any API testing application like Postman or Thunder Client. For the tutorial, I am using Thunder Client.

To generate the response, make a POST request on http://localhost:8080/api/generate with the following body,

{
  "model":"Model Name", // Replace this with the model that you pulled.
  "prompt":"Prompt", // Add your prompt here.
  "stream":false,
  "format": {} // Add the required structure here.
}

Such request will generate the response and return it in the required format. Checkout the following response when I prompted "list 5 cities of India with the primary language spoken there." to "llama3.2:latest".

How to define the required structure?

To define the required structure, you need to start by defining the type. The type can be a string, an object or an array. Following is how a type can be defined.

{
  "type": "object" // or "string" or "array"
}

Object

In case the type is defined as object, you also need to add properties and required keys with the type key. The properties key contains the list of values, required in the response along with its type. Lastly, the value of required key is an array of string which containers the list of all the keys defined in properties.

The explanation is confusing. Here is an example to understand it much better. For the prompt What is the capital of India and the primary language spoken there., following can be the structure.

{
  "type": "object",
  "properties": {
    "capital": {
      "type": "string"
    },
    "primary_language": {
      "type": "string"
    }
  },
  "required": ["capital", "primary_language"]
}

The type of properties, for eg, capital's type can also be an object or array.

Array

In case the type is defined as array, you also need to add items key. The items key contains the type of the item of the array. For instance, for the prompt List 5 cities of Canada., following can be the structure.

{
  "type": "object",
  "properties": {
    "cities": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
  },
  "required": ["cities"]
}

Again, the type of item of the array can be an object or array.

String

In case of string, you just need to define the type key.

Final Words

In conclusion, this is a great feature of generating structured responses. In my opinion it solves a big issue. I am really excited to leverage Ollama in my future projects. Follow me on Twitter and LinkedIn for more updates.