Reflection and self-correction

An agent's first attempt is a draft. Reflection is the discipline of turning that draft into a better one: the model reads its own previous attempt plus a signal about what went wrong, and produces a fix. Done right, it is how a coding agent goes from red tests to green with no human in the loop. Done wrong, it is a model cheerfully congratulating itself on a broken answer. The whole difference is one ingredient, an external verifier.

Reflection is sequential self-improvement

In the canonical anatomy of an LLM agent, planning has three sub-capabilities: decomposition (break the goal into steps), reasoning (think each step through), and reflection (critique past actions and refine).^[1] You met the first two in the previous lesson. This one is the third leg, the loop that lets an agent learn from its own mistakes within a single task, without any training.

Reflection is sequential: attempt to feedback to better attempt, each one conditioned on the last. That makes it fundamentally different from simply retrying from scratch (which throws the mistake away) and from self-consistency (which votes across independent samples and never reads a result). Reflection's power, and its danger, is that the model is grading and revising its own work.

The evaluator-optimizer loop

The simplest useful shape is a two-role workflow that Anthropic calls evaluator-optimizer: one LLM generates a candidate, another evaluates it against explicit criteria and returns feedback, and the loop repeats until the criteria are met.^[2]

The catch is in that last clause. This pattern earns its cost only when you have clear evaluation criteria and refinement demonstrably improves the result.^[2] If "the criteria" is just the same model's opinion, you have built a machine for agreeing with yourself. Hold that thought, it is the crux of this lesson.

Reflexion: write the post-mortem, then retry

Reflexion (Shinn et al., 2023) is the canonical algorithm that makes reflection concrete and gives it a memory.^[3] Four moving parts:

1. Actor. An LLM (often a ReAct-style loop) attempts the task and produces a trajectory.
2. Evaluator. Turns the outcome into a reward signal: binary/scalar, from unit tests, or from environment feedback.
3. Self-reflection model. Converts that sparse signal plus the trajectory into verbal feedback: a natural-language note on what went wrong and how to fix it.
4. Episodic memory. Stores that reflection (the working buffer holds ~3 recent ones) and prepends it as context on the next trial. Then the actor retries.

No weights are updated, the model improves purely by reading its own written post-mortems. The authors call this "verbal reinforcement learning."^[3] On code generation it reached 91% pass@1 on HumanEval, beating the then-GPT-4 baseline of 80%, precisely because the evaluator was a suite of unit tests, i.e. ground truth.^[3]

Worked example: a coding agent fixing itself

Here is the loop on one concrete task, implement median(nums). The evaluator is a unit test, so the feedback is ground truth, and the reflection is carried forward in memory.

def median(nums):
    s = sorted(nums)
    return s[len(s) // 2]      # takes the upper-middle element only

# run tests -> FAILED test_median::even
#   assert median([1,2,3,4]) == 2.5   (got 3)

def median(nums):
    s = sorted(nums)
    mid = len(s) // 2
    if len(s) % 2:                 # odd length -> single middle
        return s[mid]
    return (s[mid - 1] + s[mid]) / 2   # even -> average the middle pair

# run tests -> PASSED 4 / 4   (exit loop)

Notice what actually did the work: the failing assertion named the exact input, the exact wrong output, and (implicitly) the exact contract. The reflection is a translation of a hard signal into a plan. Swap the unit test for a vague "does this look right?" and the same loop degrades into guessing.

The honest part: reflection needs a verifier

Here is the finding the demos gloss over. When the "evaluator" is the same model with no external ground truth, reflection frequently reinforces the error instead of fixing it, the model rationalizes its first answer, or sycophantically agrees that its own reasoning was fine.^[2] Models are genuinely weak at self-correcting pure reasoning without a ground-truth signal to push against. Reflection is not magic introspection; it is a control loop, and a control loop with no sensor just amplifies noise.

Self-consistency: the parallel cousin (and why it is fading)

People often lump reflection together with self-consistency, don't. Self-consistency samples many independent chains of thought and takes a majority vote on the final answer.^[4] There is no feedback and no memory; it just bets that a correct answer is reachable by many valid paths and so it clusters. On 2022-era models it was a big win (+17.9% on GSM8K).^[4] On modern reasoning models the edge has largely eroded: gains plateau around 5 to 10 samples and are often only +1 to 3%, while cost scales linearly with the sample count.^[5]

When to reach for it

Anthropic's number-one rule applies here as everywhere: use the simplest thing that passes your eval, and add reflection only when it measurably helps.^[2] A useful checklist before you bolt on a reflect to retry loop:

1. Do you have a cheap, reliable signal? Unit tests, a validator, an environment reward, an independent judge. If not, build one first, reflection without it is theater.
2. Does iteration actually improve the metric? Measure attempt-2 vs attempt-1 on real cases. Sometimes it doesn't move, or it regresses.
3. Is the loop bounded? A hard max_trials and the stuck-loop detectors from Reflexion keep cost and runaway behavior in check.
4. Is the memory managed? Long trajectories plus accumulated reflections eat the context window, which is exactly the discipline of context engineering (next lesson), and why the buffer holds only the last few reflections.

Check yourself