The Agent Loop and Acting

Planning and decomposition

ToT, plan-and-execute, subtasks

~11 minLesson 9 of 28Written by Claude Opus 4.8

Most agents fail long tasks not because they can't do a step, but because they never decide which steps, in what order. Planning is how an agent turns a fuzzy goal into a concrete sequence, and choosing the right planning strategy is one of the highest-leverage decisions you will make.

Planning is decomposition first

Lilian Weng's canonical framing splits an LLM agent into planning + memory + tool use, all driven by the model. Planning itself has two sub-skills: task decomposition, break a big goal into smaller sub-goals the agent can actually handle, and reflection, critique past steps and correct course.^[1] This lesson is about the first; the next lesson takes the second.

The core idea in one line

A large task is hard for one forward pass but easy as a list of small ones. trades a single impossible prediction for a chain of tractable ones, and gives you a plan you can inspect, parallelise, cache, and revise.

Take a real goal: Write a research report on the state of on-device LLMs. A model asked to emit that in one shot will ramble. Decomposed, it becomes a plan you could hand to a junior, or to an executor loop:

Scope. Turn the topic into 4 to 6 concrete sub-questions (what hardware, what models, what latency/accuracy trade-offs, what is shipping in 2026).
Gather. For each sub-question, retrieve sources with a search tool; keep URLs and quotes.
Outline. Cluster findings into sections; decide the argument and the order.
Draft. Write each section from its cluster of evidence, one at a time.
Cite and edit. Attach every claim to a source, then tighten for length and flow.

Notice three things this bought us: each step has a checkable output, steps 1 and 2 can fan out in parallel, and a failure in step 2 is localised, you re-run one search, not the whole report. Everything below is a different machine for producing and running plans like this.

Four ways to plan, from cheap to heavy

Think of these as a ladder. Climb only as high as your task forces you, every rung adds tokens, latency, and failure surface.^[6]

1. Chain-of-thought, one straight line

(CoT) is the base case: the model writes intermediate reasoning before answering, decomposing inside a single trajectory. It is cheap and often enough. Its weakness is structural, it is one greedy path with no external grounding, so a wrong early step silently poisons the rest. Modern reasoning models bake long CoT in, which is why you rarely hand-write chains anymore.

2. Self-consistency, sample many, then vote

replaces greedy decoding with sampling: draw N diverse chains at temperature above 0 and take a majority vote on the final answer. The intuition, a correct answer is reachable by many valid chains, so right answers cluster. On 2022-era models it was a big win (+17.9% on GSM8K).^[2]

Diminishing returns in 2026

On modern reasoning models, majority-vote gains plateau at ~5 to 10 samples and often add only +1 to 3% while cost scales linearly with N.^[3] It also only works when the answer is a discrete, comparable value you can vote on, not free-form prose. Measure before you pay N times for it.

3. Tree-of-Thoughts, branch, evaluate, backtrack

(ToT) generalises the single chain into a search tree. Four moving parts: define what a "thought" is, generate k candidate next-thoughts per state, have the model self-evaluate each partial state (sure / maybe / impossible), then search the tree with BFS or DFS, pruning dead ends and backtracking when a branch fails. Breaking the left-to-right constraint is decisive on tasks with pivotal early choices: on Game-of-24, GPT-4 with CoT scored 4%; ToT scored 74%.^[4]

Tap a node to see what it does.

Tree-of-Thoughts on Game-of-24: propose several first moves, score each partial state, prune the impossible, backtrack on a dead end, and follow the surviving path to 24. CoT would commit to one branch and, if wrong, never recover.

ToT is powerful and expensive: every node costs LLM calls for both generation and evaluation. Reserve it for genuine search, puzzles, constraint satisfaction, planning where a greedy chain provably fails. Don't reach for it when a single call passes your eval.

Plan-and-execute, separate the planner from the doer

The three rungs above all plan inside the model, one call at a time. The pattern is architectural instead: one Planner call produces the whole multi-step plan up front, then cheap Executor steps run each task, calling tools, delegating to smaller models, without re-invoking the big planner every step.^[5]

Tap a node to see what it does.

The plan-and-execute pipeline. One expensive Planner call emits a full task list; a cheap Executor runs the steps in order, calling tools without re-consulting the planner. Only a failed step triggers a re-plan back to the planner.

Why prefer this over reactive ReAct

ReAct pays a big-model call on every step (Thought to Action to Observation, repeat). Plan-and-execute pays for the plan and any re-plans only, so it is faster and cheaper on long tasks, and forcing an explicit up-front plan often improves reasoning quality, because the model commits to structure before it gets lost in details.^[5]

Two variants push efficiency further:^[5]

ReWOO

The planner writes steps that reference previous outputs as variables (#E2), so a lightweight worker substitutes results, eliminating the re-planning LLM calls between steps.

LLMCompiler

The planner streams a DAG of tasks with dependencies; a scheduler dispatches independent tasks in parallel the moment their inputs are ready, the paper reports a ~3.6x speedup.

Up-front plan vs stay reactive, and revising

The real trade-off is commitment vs adaptivity. A plan made up front is efficient but brittle: if the environment diverges from what the planner assumed, a rigid agent barrels ahead executing a stale plan. That is exactly why the loop is Execute to Re-plan to Execute, not just Execute.^[5]

Plan up front when

Steps are largely predictable in advance
Latency or token cost really matters
Steps are independent, so parallel or cached
You want a plan a human can audit first

Stay reactive (ReAct) when

Each step depends on the last observation
The environment is noisy or unpredictable
You can't foresee the number of steps
Exceptions need per-step handling in the loop

In practice the frontier combines them: a plan-and-execute skeleton for structure and cost, with a re-plan step that behaves like ReAct whenever reality pushes back. Revision is not a fallback, it is the mechanism that makes a fixed plan safe to commit to. Just bound it: over-eager re-planning burns calls, so re-plan on failure or surprise, not on every step.

Make the plan visible, and give it a stop

Two operating rules that pay off in production. Transparency: surface the plan so a human can audit and interrupt it. Stopping condition: cap max steps / max re-plans so a stuck agent can't loop forever.^[6] And beware underthinking, reasoning models can thrash between half-formed plans without committing to any; more planning tokens only help if they are focused.^[7]

Choosing a strategy

Start with the simplest thing that passes your eval and escalate only when it fails.^[6]

Strategy	Shape	Cost	Reach for it when
Chain-of-thought	One linear path	1x	Multi-step reasoning, no external facts needed
Self-consistency	N paths, then vote	Nx	A discrete checkable answer; a few % is worth Nx^[3]
Tree-of-Thoughts	Search tree	Tree-size x	Real exploration / lookahead where greedy fails^[4]
Plan-and-execute	Plan, run, re-plan	~1 to 2 big calls	Long, predictable tool tasks; latency matters^[5]
ReAct	Loop, call per step	Steps x	Steps unpredictable; each depends on the last observation

A ladder from cheapest to heaviest. Climb only as high as your task forces you.

Check yourself

Match each planning strategy to what defines it.

drop here

one greedy line of reasoning

drop here

sample N paths, then vote

drop here

branch, score, and backtrack

drop here

one up-front plan, then run it

On a long, predictable tool task, why is plan-and-execute usually cheaper than ReAct?

Tree-of-Thoughts beats a single chain on Game-of-24 (74% vs 4%). What three moves make that possible?

What can a greedy chain not do? Try to state it, then check.

Lock it in

Decomposition is the win: a big task is hard in one pass, easy as a list of small, checkable sub-goals you can parallelise, cache, and revise.
Climb the ladder: CoT (one path) to self-consistency (vote) to Tree-of-Thoughts (branch, evaluate, backtrack). Each rung buys reach at real token cost.
Plan-and-execute separates thinking from doing: one planner call, then cheap executor steps, faster and cheaper than ReAct's per-step big-model call.
Up-front plans are brittle; the Execute to Re-plan to Execute loop is what makes committing to a plan safe. Bound the re-planning.
Start simple. Add ToT / multi-agent machinery only when a simpler approach measurably fails your eval.

Primary source

Lilian Weng, LLM Powered Autonomous Agents

The canonical decomposition of an agent into planning + memory + tools, with crisp treatments of task decomposition, chain-of-thought, and Tree-of-Thoughts in one place.

Sources