Agentic patterns · 01 of 04

Agentic patterns: Validate and retry

Treat the model’s answer as untrusted input. Check it against a type and the business rules, and when it fails, send the exact error back to the model and let it try again.

The principle

Validate and retry

A language model produces text that looks right. When that text feeds other software, looking right is not enough: a missing field, a total that does not add up or a date in the wrong format breaks whatever comes next. The remedy comes from ordinary software engineering. Define what a valid answer is, as a schema plus the rules a check can enforce, and refuse anything that fails.

The agentic part is the loop. The failed check goes back to the model as a precise error message, and the model corrects itself, usually on the next attempt. After a set number of tries the run stops with a typed failure that your code can handle, so a bad answer never slips through unnoticed.

exact errorcodeInputmodelGeneratecodeCheckcodeAccept

codemodelagentperson

The shape of the pattern. Each box can be filled by code, a model, an agent or a person; the colours show how this page fills them.

Also known as Evaluator-optimizer, with code as the evaluator (Anthropic) · Maker-checker loop (Azure Architecture Center) · Reflection (AutoGen)

Where it applies

  • Extracting fields from invoices, payslips or contracts into a system of record
  • Turning an email or a call note into a structured case or ticket
  • Calculations a model sets up and code can verify: allocations, reconciliations, splits
  • Any model output that another system reads without a person in between

Where it does not

  • Prose for a person to read, where there is nothing to check
  • Judgement calls, where a wrong answer can still pass every rule
  • Requirements you cannot express as a type or a rule

Fill the boxes

One shape, three ways to build it

The pattern exists with or without AI. The design decision is which boxes deserve a model, and what that costs in money, speed and auditability.

All code

A parser fills the fields and a validator rejects the bad ones. Retrying is pointless because the same input fails the same way, so every reject lands with a person.

Hybrid, as shown here

A model produces the answer and code checks it and returns the exact error. Each retry costs one model call, and every failed attempt is in the log.

All agents

A second agent reviews the first one’s answer. It can catch errors no rule describes. It also doubles the cost per round, and a reviewer can be talked round where a rule cannot.

See it work

Split the dinner bill

Five friends, one crumpled receipt, a group chat full of "I had the gnocchi". The agent must return a BillSplit that Python can trust: real diners, real euro amounts, and shares that add up to what the card machine asks. Watch it get the first try wrong and fix it.

These runs are simulated: Pydantic AI's FunctionModel plays the LLM with scripted answers, while the validation and retry loop is the real framework. Token counts and costs are illustrative. Live runs are coming.

Watch for. output_type=[BillSplit, ReceiptProblem] plus an @output_validator: a wrong answer never reaches your code, it goes back to the model as a retry with the exact error.

Building it

This demo runs on Pydantic AI

Pydantic AI is a Python agent framework where the answer is a typed object: if the model's output fails validation, the error goes back to the model and it tries again. Other frameworks can carry the same pattern; this is when I would pick this one.

Reach for it when

  • Code, not a person, consumes the model's answer and needs it typed
  • You can state 'correct' as a type or a rule that a check can enforce
  • Your team already models data with Pydantic and wants plain Python, any model provider

Skip it when

  • The output is prose for a human; there's nothing to validate
  • You need durable pauses, human sign-off or rewind mid-run (reach for a graph runtime)
  • A wrong answer passes every check: retries fix format and arithmetic, not judgment
Framework
pydantic-ai 2.68.0
Licence
MIT
Model
gpt-4.1-mini
Watch time
~2 min

Advice

Questions I ask before using it

  • What exactly counts as a correct answer, and can code check it?
  • How many retries is the cost worth, and what happens after the last one?
  • Do we log every failed attempt? In a regulated process the failures belong in the audit trail.
  • Which errors does the check miss, and who catches those?

Contact

Questions, ideas or a conversation?

I’m always glad to hear from people working on AI, product strategy, transformation or software.