# Large language model (LLM)

> A large language model is a neural network trained on very large amounts of text to predict the next piece of text given what came before. Everything it does…

Source: https://northerngo.com/glossary/large-language-model/
Language: en
Updated: 2026-08-28

---
**A large language model is a neural network trained on very large amounts of text to predict the next piece of text given what came before. Everything it does — answering, translating, writing code, summarising — is that one operation applied repeatedly.**

### How it is built

Almost all current models use the transformer architecture. Text is cut into tokens, each token becomes a vector, and attention layers let every position weigh every other position in the input before the model emits a probability distribution over the next token. One token is sampled, appended to the input, and the whole thing runs again.

Training happens in stages. Pre-training on a very large text corpus produces a model that continues text. Post-training — instruction tuning, then some form of preference optimisation — turns that into something that follows requests and behaves like an assistant. The assistant persona is a trained behaviour, not a property of the underlying network.

### What follows from that design

Several practical consequences fall straight out of "it predicts text":

- **It has no lookup table.** Facts are diffuse patterns in the weights, not rows in a database. This is why [hallucination](/glossary/hallucination/) is a structural property rather than a bug awaiting a patch, and why [retrieval-augmented generation](/glossary/retrieval-augmented-generation/) exists.
- **It is stateless between calls.** Any "memory" is text re-sent inside the [context window](/glossary/context-window/).
- **Its knowledge has a cutoff date**, set by when the training data was collected.
- **Output varies.** Sampling is random unless the temperature is zero, and even then hardware and batching can shift results slightly.

### What the numbers do and do not tell you

Parameter count is a capacity ceiling, not a quality score. A well post-trained 8B model beats a badly trained 70B model on most real tasks, and a large model asked a badly framed question performs worse than a small model asked a clear one. Data quality, post-training and how much the model is allowed to reason before answering all move results more than raw size.

Benchmark tables are worth even less than they look. Contamination is common, the tasks rarely resemble yours, and a two-point difference on a leaderboard is not something your users will notice. Test candidate models on twenty examples of your own work instead.

---

NorthernGo is an AI-powered platform for building production-ready web apps with zero coding. Local AI generation via WebGPU is unlimited and free, and you own all generated source code. https://northerngo.com/