DEV Community

Cover image for Ollama Cheatsheet: Running LLMs Locally with Ollama
Wanda
Wanda

Posted on • Originally published at apidog.com

Ollama Cheatsheet: Running LLMs Locally with Ollama

#llm #ai #development #devops

Ever found yourself thinking, "I wish I could run this AI model without sending my data to the cloud!" or "These API rate limits are killing my development flow!"? You're not alone! The AI world is evolving at breakneck speed, and one of the most exciting developments is the ability to run powerful language models right on your own hardware. No strings attached!

Let me introduce you to the dynamic duo that's been revolutionizing my development workflow: Ollama + LLMs(e.g.: Deepeek-R1). This combination is an absolute game-changer for anyone who wants AI power without the cloud-based headaches.

Why Local LLMs Are the Developer's New Best Friend

Let's face it - cloud-based AI services are awesome... until they're not. They come with three major pain points that make local inference increasingly attractive:

  • Privacy concerns? Gone! Your sensitive data never leaves your machine.
  • Latency issues? Eliminated! No more waiting for API calls to traverse the internet.
  • Usage quotas and unexpected bills? A thing of the past! Run as many inferences as your hardware can handle.

When I first started running DeepSeek-R1 locally through Ollama, the freedom was almost intoxicating. No more watching my token count like a nervous accountant! 😅

Getting Ollama Up and Running in Minutes

Installation is refreshingly straightforward - none of that "dependency hell" we've all come to dread in the dev world:

# After installation, start the Ollama server with:
ollama serve
Enter fullscreen mode Exit fullscreen mode

This launches Ollama as a service listening on localhost:11434. Keep this terminal window running, or if you're like me and hate having extra terminals cluttering your workspace, set it up as a background service.

What Your Machine Needs to Handle the AI Beast

For DeepSeek-R1 to run smoothly:

  • Minimum: 8GB RAM, modern CPU with 4+ cores
  • Recommended: 16GB+ RAM, NVIDIA GPU with 8GB+ VRAM
  • Storage: At least 10GB free space for the base model

I started on a modest setup and let me tell you... watching my CPU fans spin up to aircraft takeoff levels was quite the experience! Upgrading to a decent GPU made a world of difference.

Model Management Made Simple

Before diving into the AI playground, let's see what's available:

ollama list
Enter fullscreen mode Exit fullscreen mode

Ready to pull DeepSeek-R1? It's as simple as:

ollama pull deepseek-r1
Enter fullscreen mode Exit fullscreen mode

Ollama thoughtfully provides different model sizes to match your hardware capabilities:

# For machines with limited resources:
ollama pull deepseek-r1:7b

# For more powerful setups seeking enhanced capabilities:
ollama pull deepseek-r1:8b
Enter fullscreen mode Exit fullscreen mode

Chatting With Your Local AI Brain

Here's where the magic happens! Launch an interactive chat session:

ollama run deepseek-r1
Enter fullscreen mode Exit fullscreen mode

This opens a real-time conversation where you can explore the model's capabilities. It's like having a super-smart (but occasionally confused) colleague sitting right next to you!

Need a quick answer without the full chat experience?

ollama run deepseek-r1 "Explain quantum computing in simple terms"
Enter fullscreen mode Exit fullscreen mode

One of my favorite features is processing text directly from files:

cat complex_document.txt | ollama run deepseek-r1 "Summarize this text"
Enter fullscreen mode Exit fullscreen mode

This has saved me hours of reading through dense documentation and research papers!

Fine-tuning Your AI's Personality

Want DeepSeek-R1 to be more creative? More factual? You can dramatically alter its behavior through parameter adjustments:

# For creative, varied outputs:
ollama run deepseek-r1 --temperature 0.8

# For factual, deterministic responses:
ollama run deepseek-r1 --temperature 0.1
Enter fullscreen mode Exit fullscreen mode

Pro tip: Lower temperature values (0.1-0.3) are fantastic for coding tasks, while higher values (0.7-0.9) produce more creative content. I learned this the hard way after getting some... let's just say "imaginative" code that definitely wouldn't compile! 🤦‍♂️

Taking It to the Next Level: API Integration

While the command line is great for experimentation, real-world applications need API access. Ollama's REST API is refreshingly simple:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "deepseek-r1",
  "prompt": "Write a function that calculates fibonacci numbers"
}'
Enter fullscreen mode Exit fullscreen mode

For streaming responses (ideal for chat interfaces):

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "deepseek-r1",
  "prompt": "Write a story about a robot learning to love",
  "stream": true
}'
Enter fullscreen mode Exit fullscreen mode

Powerful LLMs Deserve Powerful API Testing

When building applications that integrate with local LLMs like DeepSeek through Ollama, you'll inevitably face the challenge of debugging streaming AI responses. That's where Apidog truly shines!

Unlike generic API tools that just dump raw text at you, Apidog's specialized debugging features for AI endpoints are mind-blowing. When debugging endpoints for AI with LLMs deployed locally with Ollama, Apidog can automatically merge message content and display responses in natural language. It supports reasoning models such as DeepSeek R1, allowing you to visualize the deep thought process of your AI model in real-time.

Click to check out this beauty in action here.

Get Started with Apidog Now

I mean, just look at that! Being able to see the token-by-token generation gives you unprecedented visibility into how your model thinks. Whether you're building a chatbot, content generator, or AI-powered search, this level of insight is invaluable.

Setting up Apidog to test Ollama is straightforward:

  1. Create a new HTTP project in Apidog
  2. Add an endpoint with the URL http://localhost:11434/api/generate
  3. Set up a POST request with the JSON body: 
{
  "model": "deepseek-r1",
  "prompt": "Explain how to implement a binary search tree",
  "stream": true
}
Enter fullscreen mode Exit fullscreen mode
  1. Send the request and watch the magic happen!

I've personally found this combination to be revolutionary for local LLM development. Being able to see exactly how the model constructs its responses has helped me fine-tune prompts in ways I never could before. It's like having X-ray vision into your AI's brain!

Real-World Applications That Will Blow Your Mind

DeepSeek-R1 excels in various practical scenarios:

Content Generation That Doesn't Suck 

ollama run deepseek-r1 "Write a professional blog post about sustainable technology practices"
Enter fullscreen mode Exit fullscreen mode

Information Extraction That Actually Works 

ollama run deepseek-r1 "Extract the key points from this financial report: [report text]"
Enter fullscreen mode Exit fullscreen mode

Code Generation That Makes You Look Like a Genius 

ollama run deepseek-r1 "Write a Python function that implements a Red-Black tree with insertion and deletion"
Enter fullscreen mode Exit fullscreen mode

I once had a tight deadline for implementing a complex algorithm, and DeepSeek-R1 not only generated the code but also explained the logic so well that I could confidently modify it for our specific needs. My team thought I'd pulled an all-nighter... little did they know! 😎

When Things Go Sideways: Troubleshooting

If you encounter out-of-memory errors (and you probably will at some point):

  • Try a smaller model variant (7B instead of 8B)
  • Reduce the context window size with --ctx N (e.g., --ctx 2048)
  • Close those 47 browser tabs you've been "meaning to read later"

For API connection issues:

  • Ensure Ollama is running with ollama serve
  • Check if the default port is blocked
  • Verify firewall settings if connecting from another machine

And when debugging API responses seems impossible, remember that Apidog's visualization capabilities can help identify exactly where things are going wrong in the model's reasoning process.

The Bottom Line: Local AI Is Here to Stay

Ollama with DeepSeek-R1 represents a significant step toward democratizing AI by putting powerful language models directly in developers' hands. The combination offers privacy, control, and impressive capabilities—all without reliance on external services.

As you build applications with these local LLMs, remember that proper testing of your API integrations is crucial for reliable performance. Tools like Apidog can help visualize and debug the streaming responses from Ollama, especially when you're building complex applications that need to process model outputs in real-time.

Whether you're generating content, building conversational interfaces, or creating code assistants, this powerful duo provides the foundation you need for sophisticated AI integration—right on your own hardware.

Have you tried running LLMs locally? What's been your experience with tools like Ollama and Apidog? Drop your thoughts in the comments below—I'd love to hear about your local AI adventures!

Top comments (0)

Read next

scovl profile image

Análise Comparativa: Aider vs. PMD vs. Semgrep

Vitor Lobo -

aairom profile image

My experience and feedback on Hugging Face (smolagent) Agent Course (so far)

Alain Airom -

angelinajasper profile image

Benefits and Challenges of the Internet of Things (IoT)

Angelina Jasper -

mrunmaylangdb profile image

Choose the right AI model: Comparision of gpt-4o, claude and gemini using LangDB

Mrunmay Shelar -