Local LLM Memo

Table of contents

• Tool
• How to run LLM localy
• How to use LLM for personal research agent
• Create API Server

Tool

Ollama

It can run with large language models (LLMs).
Although each models has different interface, it has unified interface to use lcoal LLM, so you can run LLMs easily.

LangChain

LangChain is a composable framework to build with LLMs.
Langchain

LangGraph

LangGraph is the orchestration framework for controllable agentic workflows.
It defines state and create complex workflow.

Duckdockgo-search

This is a search engine that protects user privacy.

DuckdockGo

How to run LLM locally

I used Ollama to run LLMs locally.

Download Ollama

Download Ollama from the official website.
Ollama

Install

Install Ollama following the installer.

Download LLM

Download the LLM you want to download.

ollama pull <model_name>

Run LLM

Run the LLM

ollama run <model_name>

How to use LLM for personal research agent

I referred the below technical article.
How to build a Interactive Personal AI Research Agent with Llama 3.2 : A Step-by-Step Guide using LangChain and Ollama

Create API Server

I built API server using render and Flask.