Prompting and in-context learning

The weights are frozen - yet the model "learns" your task from a handful of examples you drop into the prompt, and reasons better when you tell it to think out loud.

After pre-training and alignment, a large language model's weights are fixed. You cannot retrain it every time you have a new task - but you don't have to. By writing a good prompt, you steer the frozen model at inference time. Show it two or three worked examples and it picks up the pattern on the spot; that is in-context learning. Ask it to "think step by step" and a problem it was botching suddenly comes out right. No gradient descent, no fine-tuning - just the right text in the context window, consumed by a single forward pass.

The prompting ladder: zero, one, few

Everything hangs on how many worked examples (called shots) you place before your real question.

• Zero-shot - just the instruction and the question. "Classify the sentiment: 'the film dragged.'" The model relies entirely on what pre-training baked in.
• One-shot - a single worked example first, so the model sees the exact input to output format you expect.
• Few-shot - several examples. Now the model has enough to infer the whole pattern, and accuracy usually climbs sharply over one-shot.

A system prompt sits above all of this - a persistent instruction ("You are a terse SQL expert; answer in one query") prepended to every turn, setting role and rules before the conversation even starts.

In-context learning: the weights never move

Call the frozen parameters $\theta$. Given $k$ demonstration pairs and a new input $x$, the model just computes a conditional probability over the next tokens and picks the most likely completion:

Feel it: the prompt-builder sandbox

The task is grade-school math. Drag example cards from the bank into the context to add shots, and watch held-out accuracy climb (few-shot > one-shot > zero-shot). Then flip chain-of-thought on: the same frozen model, now shown its reasoning, jumps from mostly-wrong to mostly-right - while the token counter climbs, because every example and every reasoning step spends context-window budget.

Chain-of-thought: make it show its work

Ask a model "Roger has 5 balls, buys 2 cans of 3 - how many now?" and a terse model blurts a wrong number. Prepend "Let's think step by step" (or show exemplars that reason), and it writes "5 + 2×3 = 5 + 6 = 11" and lands on 11. The extra tokens are not decoration: each intermediate result is written into the context, so the next step can attend to it instead of solving everything in one leap. The cost is real - a longer prompt and a longer answer eat more of the context window - but on multi-step reasoning the accuracy gain is large, which is exactly what the chain-of-thought paper measured.

# The weights (theta) are frozen - this all happens at inference.
examples = [
    ("4 cars, each with 4 wheels. How many wheels?", "16"),
    ("20 apples, ate 8, bought 12. How many now?",   "24"),
]

def build_prompt(query, cot=False):
    shots = ""
    for q, a in examples:          # each pair is one "shot"
        if cot:
            shots += f"Q: {q}\nA: Let's think step by step. ... The answer is {a}.\n\n"
        else:
            shots += f"Q: {q}\nA: {a}\n\n"
    tail = "A: Let's think step by step." if cot else "A:"
    return f"{shots}Q: {query}\n{tail}"

answer = model.generate(build_prompt(query, cot=True))  # theta unchanged

Check yourself