Large language models: the autocomplete that grew up

You type "Assalam o" into WhatsApp and your phone suggests "alaikum". You type "I'll be there in" and it offers "5 minutes". Your keyboard has learned which word usually comes next.

Now imagine a keyboard that read a large part of the internet, millions of books, every Wikipedia article, endless code, forums and manuals. Not just the next word for greetings, but the next word for physics, poetry, tax law and Python.

That is, at its heart, a large language model (LLM). ChatGPT, Claude, Gemini and Llama are all built on this idea.

Stage 1: the endless fill-in-the-blank

The first stage of training is called pre-training. The model, a giant Transformer (Part 7), is shown text with the next token hidden and asked to guess it. Then the next. Then the next. Trillions of times.

The capital of Pakistan is ___

Each wrong guess creates a loss, the loss is sent backwards (Part 1), and the weights take a small step downhill (Part 2).

Something remarkable happens along the way. To get good at predicting the next word in a chemistry textbook, the model has to pick up some chemistry. To predict the next line of code, it has to pick up programming patterns. Knowledge and skills appear because they help with prediction.

Stage 2: teaching it to be helpful

A model that has only been pre-trained is a brilliant mimic, but not an assistant. Ask it a question and it might continue with three more questions, because that's what the page it's imitating looked like.

So there is a second stage. The model is fine-tuned on many examples of good conversations: a question, and a helpful, honest answer. Then people (and other models) compare pairs of answers and say which is better, and the model is tuned towards the preferred ones. One well-known version of this is reinforcement learning from human feedback (RLHF).

It is the difference between a gifted student who has read every book in the library, and the same student after training at a job where they learned to listen, answer clearly and admit what they don't know.

How an answer is written

When you ask a question, the model:

1. Splits your message into tokens (Part 6).
2. Runs them through its layers of attention.
3. Produces a probability for every possible next token.
4. Picks one, adds it to the text, and repeats, one token at a time, until the answer is done.

A setting called temperature controls how adventurous the choice is. Low temperature picks the most likely token almost every time: predictable, good for facts. Higher temperature allows less likely choices: more varied, good for brainstorming.

Why it sometimes makes things up

Here is the honest part. The model was trained to produce text that sounds right. Most of the time, sounding right and being right go together. But not always.

Think of the confident uncle at a wedding who always has an answer. Ask him the price of gold, the best cricket team of 1992 or the side effects of a medicine, and he will answer smoothly. Usually he is right. Sometimes he is completely wrong, and he sounds exactly as confident.

When an LLM states something false with confidence, it's called a hallucination. It happens most with rare facts, very recent events, precise numbers and anything specific to your business, which the model never saw.

The maths, briefly

A language model learns the probability of each token given all the tokens before it, so the probability of a whole text is a product:

$$P(x_1, \dots, x_T) = \prod_{t=1}^{T} P(x_t \mid x_{<t})$$

Pre-training minimises the cross-entropy loss, the model's average surprise at the real next token:

$$L = -\frac{1}{T} \sum_{t=1}^{T} \log P_\theta(x_t \mid x_{<t})$$

The final layer gives a score $z_i$ to every token in the vocabulary, and temperature $\tau$ reshapes those scores before one is picked:

$$P(i) = \frac{e^{z_i / \tau}}{\sum_{j} e^{z_j / \tau}}$$

A small $\tau$ sharpens the distribution towards the top choice; a large $\tau$ flattens it.

What to remember

• An LLM is a huge Transformer trained to predict the next token.
• Pre-training gives it knowledge and skills; fine-tuning and feedback make it a helpful assistant.
• It writes one token at a time, choosing from probabilities.
• It can sound right while being wrong, so important facts should come from trusted sources.

So how do you stop the confident uncle from guessing? You hand him the right file before he answers. That's Part 9.