How to Install and Set Up Ollama: Step-by-Step Guide

Ollama makes it incredibly easy to run large language models on your own machine. Here's how to get started on any platform. ## What is Ollama? Ollama is a free, open-source tool that lets you download and run LLMs locally with a simple command-line interface. It handles all the complex setup—model downloading, quantization, and GPU acceleration—automatically. ## Installation ### macOS ```bash # Download the installer from ollama.ai or use Homebrew: brew install ollama ``` Or download the macOS app from [ollama.ai](https://ollama.ai) and drag it to your Applications folder. ### Windows Download the Windows installer from [ollama.ai](https://ollama.ai) and run it. Ollama will be available in your terminal after installation. ### Linux ```bash curl -fsSL https://ollama.ai/install.sh | sh ``` ## Running Your First Model Once installed, open a terminal and run: ```bash ollama pull llama3.2 ollama run llama3.2 ``` This downloads and starts an interactive chat session with Meta's Llama 3.2 model. ## Popular Models to Try - `llama3.2` - Meta's latest 3B/11B model - `mistral` - Efficient 7B model - `mixtral` - Mixture of experts 8x7B - `deepseek-r1` - DeepSeek's reasoning model - `phi3` - Microsoft's compact model - `gemma2` - Google's lightweight model ## Using Ollama as an API Ollama includes an OpenAI-compatible API server: ```bash ollama serve ``` Then connect from any OpenAI-compatible client: ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1") response = client.chat.completions.create( model="llama3.2", messages=[{"role": "user", "content": "Hello!"}] ) ``` ## Tips - Use `ollama list` to see downloaded models - Use `ollama rm <model>` to remove models - Models are stored in `~/.ollama/models/` - For GPU acceleration on NVIDIA, install CUDA toolkit ## Troubleshooting **Q: Ollama is slow on my machine.** A: Try smaller models like `phi3` or `llama3.2:1b`. Ensure you have sufficient RAM. **Q: How do I update Ollama?** A: On macOS/Linux: re-run the installer. On Windows: download the latest installer. **Q: Can I use Ollama offline?** A: Yes! Once a model is downloaded, no internet connection is needed.