# Ollama

> Ollama is an open-source tool that runs language models on your own computer and exposes them over a local HTTP API. It handles downloading, quantized model…

Source: https://northerngo.com/glossary/ollama/
Language: en
Updated: 2026-08-28

---
**Ollama is an open-source tool that runs language models on your own computer and exposes them over a local HTTP API. It handles downloading, quantized model files and GPU setup, so using a local model becomes one command instead of a build process.**

### How it is used

Install it, then pull and run a model:

```bash
ollama pull llama3.1
ollama run llama3.1
```

A background service listens on `http://localhost:11434`. That is the part that matters for developers: any application on the machine can call it, and Ollama also exposes an OpenAI-compatible endpoint, so most existing client libraries work by changing the base URL and passing a dummy API key.

Models are distributed as [quantized](/glossary/quantization/) GGUF files, most commonly 4-bit, and are pulled from a registry by name and tag. A `Modelfile` lets you package a base model together with a system prompt and sampling parameters under your own name — useful for pinning a configuration your whole team uses.

Underneath it is llama.cpp, which is why it runs on Apple Silicon, NVIDIA and AMD GPUs and falls back to CPU. CPU-only works and is slow; expect a few tokens per second on a large model.

### Where it fits

Ollama is the standard answer for [local AI](/glossary/local-ai/) when the model should run on the machine but outside the browser. Compared with a browser runtime on [WebGPU](/glossary/webgpu/), it gives you larger models, a bigger context window and better speed, at the cost of requiring an install. NorthernGo can connect to a local Ollama instance for exactly that reason: it is the option for people who want a guarantee that nothing leaves the machine.

### What to watch out for

- **A web page cannot call localhost by default.** Ollama's CORS policy blocks browser origins until you allow them, typically by setting `OLLAMA_ORIGINS` before starting the service. This is the single most common reason a browser integration silently fails, and it is a deliberate protection rather than a bug.
- **Do not expose it to the network.** The API has no authentication. Binding it to a public address hands anyone your GPU, and worse.
- **Memory is the constraint.** A model that does not fit in VRAM spills to system RAM and becomes dramatically slower, or fails outright.
- **A model name is not a guarantee.** The default tag is usually a 4-bit quantization of an instruction-tuned variant, which is not the same as the full-precision model of that name. Check the tag when comparing quality against a hosted provider.

---

NorthernGo is an AI-powered platform for building production-ready web apps with zero coding. Local AI generation via WebGPU is unlimited and free, and you own all generated source code. https://northerngo.com/