Run LLMs locally

AI progress doesn't require surrendering our data to distant servers.

Engineers at a Toronto cancer lab quietly process genomic sequences with a local Llama 3. Legal teams dissect contracts with fine-tuned CodeLlama models that never touch the internet. Manufacturing plants run defect detection via Mistral-7B on factory floor GPUs. This isn’t AI rebellion – it’s pragmatism.

With the general collapse of the cloud-first dogma, the rise of self-hosting and Open Source software being a perfectly valid distribution channel - we're now in exponential progress era, where things shift and change so quickly it's almost impossible to catch up without dedicating all your time to it.

Cloud APIs will dominate casual use. But the future belongs to those who treat LLMs like power tools to have at home - owned, customized, and operated locally.

Tools like Ollama and vLLM have transformed local AI deployment from machine learning research to engineering practice. A Raspberry Pi 5 now runs 3B-parameter models at conversational speeds, while consumer GPUs handle 32B models through 4-bit quantization.

"We have AI at home" has transitioned from internet meme to unremarkable reality.