Do you want to have your own ChatGPT? Do you care about privacy? You don't want to pay for the premium version? or you want to use uncensored models. Or even train a model with your company documents.
For these reasons it is a good idea to set up your own LLM model server.
This is an example:
What is OpenLLaMA?
LLama 2 is an LLM (Large Language Model Meta AI)
LLama Meta
What do we need?
Install OpenLLAMA, OpenWebUI and Docker
Installing Ollama
Ollama have support for Windows, Linux and Mac
I recommend minimum 16Gb of RAM.
https://ollama.com/download
For linux setup
curl -fsSL https://ollama.com/install.sh | sh
Download models
ollama pull llama2-uncensored
Run model
ollama run llama2-uncensored
Install Open Web UI
Before install OpenWebUI we need to have installed Docker and running.
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_API_BASE_URL=http://127.0.0.1:11434/api --name open-webui --restart always ghcr.io/open-webui/open-webui:main
docker run -d -p 3000:8080 -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama
Top comments (0)