We’ve just rolled out Potpie’s multi-LLM feature, enabling developers to seamlessly integrate and switch between multiple LLMs, including Ollama, a powerful locally hosted AI solution.
With this update, developers can now use any LLM of their choice, including Ollama, to build and deploy AI Agents with greater flexibility.
This feature is powered by LiteLLM, a lightweight framework that standardizes API calls across multiple AI providers. It allows Potpie to efficiently route requests to different LLMs while handling request formatting and model-specific optimizations behind the scenes.
This feature is available in our latest release (v0.1.1), which you can access here: Potpie v0.1.1 Release
Using Ollama with Potpie to Build AI Agents
Ollama is a locally hosted LLM solution that packages large language models into self-contained bundles, including weights and configuration, simplifying deployment. Ollama allows developers to run models entirely offline, providing advantages like:
- Privacy & Security – No external API calls, ensuring data remains on your machine.
- Lower Latency – Faster response times without network dependency
- Cost Efficiency – No API usage fees, making it ideal for frequent queries.
- Customizability – Supports fine-tuning and model modifications for specific tasks.
Ollama leverages optimized libraries like llama.cpp for efficient inference on CPUs and GPUs, ensuring smooth performance even on resource-limited devices. It also features a straightforward command-line interface for managing models, making it developer-friendly and highly customizable.
Setting Up Potpie with Ollama
With this update, Potpie AI Agents can now seamlessly leverage Ollama models in a local setup, giving developers complete control over configurations. Setting up Potpie locally is straightforward, just follow our Getting Started Guide to install and configure it on your machine.
Checkout our Github Repo and give us a star⭐
App | Documentation | API Reference | Chat with 🥧 Repo
Prompt-To-Agent: Create custom engineering agents for your code
Potpie is an open-source platform that creates AI agents specialized in your codebase, enabling automated code analysis, testing, and development tasks. By building a comprehensive knowledge graph of your code, Potpie's agents can understand complex relationships and assist with everything from debugging to feature development.
📚 Table of Contents
- 🥧 Why Potpie?
- 🤖 Our Prebuilt Agents
- 🛠️ Tooling
- 🚀 Getting Started
- 💡 Use Cases
- 🛠️ Custom Agents
- 🗝️ Accessing Agents via API Key
- 🎨 Make Potpie Your Own
- 🤝 Contributing
- 📜 License
- 💪 Contributors
🥧 Why Potpie?
- 🧠 Deep Code Understanding: Built-in knowledge graph captures relationships between code components
- 🤖 Pre-built & Custom Agents: Ready-to-use agents for common tasks + build your own
- 🔄 Seamless Integration: Works with your existing development workflow
- 📈 Flexible: Handles codebases of…
Once installed, integrating an LLM requires defining four key parameters:
- LLM_PROVIDER – The AI provider (e.g., OpenRouter, Ollama, etc.)
- LLM_API_KEY – The API key for the selected provider
- LOW_REASONING_MODEL – The model used for lightweight tasks
- HIGH_REASONING_MODEL – The model used for complex reasoning
To configure Ollama, use the following setup:
LLM_PROVIDER=ollama
LLM_API_KEY=ollama
LOW_REASONING_MODEL=ollama_chat/qwen2.5-coder:7b
HIGH_REASONING_MODEL=ollama_chat/qwen2.5-coder:7b
Ollama supports multiple models, and you can choose from any reasoning models it provides based on your use case.
Why Use Ollama Locally with Potpie?
Running Potpie locally with Ollama provides the ultimate flexibility and control over AI Agent development. Developers can:
- Experiment with any AI model, including open-source, fine-tuned, or custom models.
- Optimize inference costs, since Ollama runs entirely on local hardware without API fees.
- Enhance security and privacy, as no data is sent to external providers
- Improve response times by eliminating network latency.
For those using default LLMs supported by Potpie (GPT, Gemini, Claude, etc.), reasoning models don’t need to be specified, Potpie automatically selects the most suitable option. However, with Ollama, developers can customize their model selection for better performance and task-specific tuning.
Other Supported LLM Models
While Ollama is a powerful choice for local AI inference, Potpie also supports a variety of cloud-based LLMs for those who prefer API-based solutions. You can integrate any supported model using your own API keys or generate a new Potpie key.
Default LLMs Supported by Potpie:
- OpenAI’s GPT – A versatile model with strong natural language understanding and code generation capabilities.
- Google’s Gemini – Designed for multimodal tasks with advanced reasoning and contextual awareness.
- Anthropic’s Claude – Excels in code understanding and generation, with advanced problem-solving capabilities for technical tasks.
- Meta's Llama – An open-source foundation model that offers strong performance with greater customization and deployment flexibility.
- DeepSeek – Cost-efficient models with strong reasoning abilities, ideal for complex problem-solving.
We are continuously expanding support for additional open-source and proprietary LLMs based on community feedback and advancements in AI.
Get Started with Potpie & Ollama Today!
Ready to build privacy-first, cost-efficient AI Agents with Ollama? Download Potpie v0.1.1 and start experimenting with local LLM inference today! Get Potpie v0.1.1 Now
Top comments (0)