I have been experimenting with various Large Language Models (LLMs) for things like proof reading, code suggestions, learning Japanese using platforms like ChatGPT and Claude AI. However, I keep hitting the daily limits and I have to go to different websites to use their chat interfaces. Their subscriptions are not cheap.
I then tried some local LLMs, using Ollama. Ollama doesnt have an official chat interface, and I found Open WebUI. It is super easy to set up, and supports both OpenAI API and Ollama out of the box using your API keys. Although there are still costs associated with purchasing credit to use the API keys, these expenses are significantly lower than the subscriptions. Since I already have credits for some of my personal projects so this is a plus.
Open WebUI can be extended use Anthropic API keys (and other LLM API keys) using Pipelines.
Step 1: Install Ollama (Optional)
Ollama is an open source tool that enables us to run various local LLMs on our own machines. To get started, download the installer from https://ollama.com/. Once install, use the CLI to pull a model. For reference, the available models are listed here.
For example, to pull the model llama3.2, use this command ollama pull llama3.2 in your terminal.
Step 2: Set Up Open WebUI
The setup is straightforward using docker (recommended). The Open WebUI Getting Started Page also has instructions for alternative installation methods.
Once the setup process is completed, Open WebUI will be available at http://localhost:3000
Navigate to Admin Panel > Settings > Connections to enter your openAI API Key
Step 3: Install Pipelines (Optional)
Pipelines are like plugins for Open WebUI. They enable us to perform actions using non-OpenAI LLM providers, and much more.
Again, setting up Pipelines is pretty straightforward using docker. Instructions are available here.
The Docker Desktop containers page should look like this (Im running Open WebUI on port 3080 instead of 3000).
Navigate to Admin Panel > Settings > Connections > OpenAI API , and add a new entry:
API Base URL: http://localhost:9099 API KEY: 0p3n-w3bu!
If Open WebUI is also running in a docker container, use the following configuration:
Base URL: http://host.docker.internal:9099API KEY: 0p3n-w3bu!
The Connections section should now look like this.
We will add the Anthropic Manifold Pipeline as an example. All the official pipelines are available here. Its important to only fetch pipelines from sources you trust.
Paste this URL into the Install from Github URL and then click the install button:
https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/anthropic_manifold_pipeline.py
After installing the pipeline, enter your Anthropic API Key and click Save.
Now I can pick Anthropic, OpenAI or Ollama models using the same app.
Open WebUI is a powerful tool that can be customized to meet different needs. To explore the various features it offers and how they can enhance your experience, be sure to check out their features page here.
Top comments (1)
Thanks for writing this post. I started using anthropic in this way.