What is an LLM?

⏱ 25 min

Hook

You ask a chatbot to write a haiku and it does. You ask it to explain a tax form and it does that too. It feels like the model understands both tasks. But under the surface it is running the same tiny operation for each — guessing the next chunk of text, over and over. Once you see that one mechanism, the model stops being magic and starts being something you can predict.

Concept

A large language model (LLM) is a program that, given some text, predicts what text is likely to come next. That is the entire core idea. Everything else — answering questions, writing code, holding a conversation — is that one prediction applied repeatedly.

Concretely, the model works with tokens: short chunks of text, roughly a word or a piece of one (you will study these closely in the next lesson). Given the tokens so far, the model outputs a probability distribution over every token it knows — a number for each possible next token saying how likely it is. Written as math, for a sequence of tokens $w_1, w_2, \dots, w_{t-1}$ the model estimates:

$$P(w_t \mid w_1, w_2, \dots, w_{t-1})$$

The analogy people reach for is autocomplete on your phone, and it is a fair start. The precise difference is scale and context: phone autocomplete looks at the last word or two, while an LLM conditions its guess on thousands of prior tokens at once, which is what lets it stay on topic across a long answer.

To turn one prediction into a paragraph, the model runs a loop. It predicts a distribution, picks a token from it, appends that token to the text, and feeds the whole thing back in to predict the next one. This one-token-at-a-time loop is called autoregressive generation.

Two words for two very different phases matter here:

Phase	When it happens	What changes
Training	Once, before release	The model's weights are adjusted from huge text
Inference	Every time you use it	Nothing is learned; weights are fixed and read

Training is where the model learns. It reads an enormous amount of text and adjusts billions of internal numbers — its weights — so that its next-token predictions match what really came next in the training data. This is expensive and happens once, before you ever touch the model.

Inference is what happens when you send a prompt. The weights are frozen. The model reads your text and runs the prediction loop using those fixed weights. It does not learn from your conversation and does not remember it after the session ends. When people say a model "learned" something from chatting with them, that is a misunderstanding — nothing was written back into the weights.

So what is the "model" itself? Strip away the interface and a model is two things: an architecture (the fixed wiring that defines how tokens flow through the network, covered in Module 3) and a set of weights (the learned numbers that fill that wiring in). A file of weights plus the code to run the architecture is the whole model. There is no database of facts and no lookup table of answers inside — only weights that, run forward, produce a next-token distribution.

🧠 Knowledge check

1. At its core, what does a large language model compute?

a) The single correct answer to any questionb) A probability distribution over the next token given the preceding tokensc) A database lookup of stored factsd) A grammar-checked rewrite of its input

2. You have a long chat with a hosted model. What happens to its weights during that chat?

a) They update after every message so the model learns from youb) They stay fixed; inference reads the weights but never changes themc) They are deleted and rebuilt for each new promptd) They are copied to your device for the session

Worked example

Let me make the prediction loop concrete by simulating it in plain Python. This is not a real LLM — it is a tiny stand-in that shows the shape of what a model does: keep a distribution over next tokens, pick one, append it, repeat.

import random

# A toy "model": for a given context word, how likely each next word is.
# A real LLM has billions of weights instead of this hand-written table,
# but the interface is the same — context in, next-token distribution out.
next_token_probs = {
    "the": {"cat": 0.6, "dog": 0.3, "sky": 0.1},
    "cat": {"sat": 0.7, "ran": 0.3},
    "sat": {"down": 0.8, "still": 0.2},
    "dog": {"ran": 0.9, "sat": 0.1},
    "ran": {"away": 1.0},
}


def predict_next(context_word):
    """Return the next-token distribution for the last word of context."""
    return next_token_probs.get(context_word, {})


def generate(start, max_tokens=6):
    """Autoregressive loop: pick a token, append it, feed it back in."""
    tokens = [start]
    random.seed(0)  # deterministic for a reproducible demo
    for _ in range(max_tokens - 1):
        dist = predict_next(tokens[-1])
        if not dist:
            break
        # Sample the next token according to its probability.
        choices, weights = zip(*dist.items())
        nxt = random.choices(choices, weights=weights, k=1)[0]
        tokens.append(nxt)
    return tokens


print(generate("the"))

['the', 'dog', 'ran', 'away']

Read what happened against the two ideas from the concept. First, inference only: next_token_probs is the frozen "weights" — the loop reads it but never edits it, exactly as a real model reads fixed weights during inference. Second, autoregressive generation: each step used only the last produced token to pick the next, appended it, and looped. Change the seed and you get a different but still plausible sentence, because the model samples from a distribution rather than emitting one fixed answer. That single fact — sampling from a distribution — is why the same prompt can give different outputs, which you will study directly in the sampling module.

Hands-on

Now watch a real model do this on text you control. The exercise below opens in the Prompt Playground with a short prompt and the model's next-token view turned on, so you can see the distribution and how continuing changes it.

▶ Prompt Playgroundga101-next-token-explorer

Try it in the Prompt Playground ↗

Do this: submit the starter prompt, note the top few candidate next tokens and their probabilities, then accept one token and observe how the distribution shifts for the token after it. Success criteria: you can point to the predicted distribution, name the token that was chosen, and explain in one sentence why the next distribution changed. The Playground records your run so you can replay it.

Recap

• You can explain an LLM as a next-token predictor that builds text with an autoregressive loop.
• You can distinguish training (weights change, happens once) from inference (weights frozen, happens every use).
• You can describe a "model" as an architecture plus learned weights, with no fact database inside.

Next up, you will open up the token itself: how text is split into tokens, why that split explains a surprising amount of model behavior, and how it sets the limits and cost you will manage in every later course.

🧠 Knowledge check

1. Why can the same prompt produce different outputs on different runs?

a) The model retrains itself between runsb) A token is sampled from a probability distribution, so choices can differc) The internet content behind the answer changedd) The prompt is secretly edited each time

2. Which statement best describes what a trained model file contains?

a) A searchable index of every document it readb) Learned weights that, run through a fixed architecture, produce a next-token distributionc) A list of question-and-answer pairsd) A copy of the training data compressed for lookup

📌 Key takeaways

• An LLM predicts the next token from the tokens so far, and builds text by looping that prediction (autoregressive generation). - Training changes the weights once; inference keeps them frozen and never learns from your chat. - A model is an architecture plus learned weights — not a database of facts — and it samples outputs, which is why they vary.