Episode 07 04 — A structured data extraction pipeline ( CCA- F scenario 6)
Take a messy stack of invoices — different formats, EU decimal commas, DD/MM dates, the occasional
smudged total — and turn them into clean, trustworthy rows in a database. At scale. Without a human
reading each one.
This is where four earlier episodes converge into a single pipeline. You've seen every piece. Here's
how they snap together into something you'd actually ship.
THE PIPELINE SHAPE. A document comes in; structured data comes out — through extract, validate, and
route stages. RAW DOC → EXTRACT → VALIDATE → ROUTE → DATABASE.
STAGE 1 — FORCED SCHEMA (callback 04-01). A Pydantic Invoice model becomes a tool's input_schema,
and tool_choice forces the call. The model MUST return the shape. No hopeful JSON, no "please respond
in this format and pray."
FEW-SHOT TO LOCK CONVENTIONS (callback 04-02). Pin two or three examples — "1.234,56 €" → 1234.56,
"31/12/2025" → "2025-12-31" — so the EU formats come out right instead of silently US-normalized. A
decimal comma read as a thousands separator is a 1000x error hiding in plain sight.
VALIDATE + RETRY, WITH A CEILING (callback 04-01). An ambiguous invoice fails the schema. Catch the
error, append it back as a turn, retry — and stop at max 1. The ceiling matters: a genuinely bad doc
can't be allowed to burn 20 calls trying.
CONFIDENCE ROUTING (callback 04-05). Each output row carries a confidence field. A dashed line at
0.70 splits the flow: at or above, the row flows green to the database; below, it diverts amber to a
human review tray. You're not trusting every extraction equally — you're routing by certainty.
CACHED CASE-FACTS (callback 06-01). Pin a vendor-policy block — the decimal rules, the date rules,
"never invent a value" — with a cache_control marker. It's the same stable prefix on every doc, so
it comes back ~90% cheaper on repeat. Cheap AND consistent.
RUN IT AT SCALE. Pour a thousand invoices through. A counter shows roughly 92% auto-accepted straight
to the database, ~8% diverted to a human. That split IS the leverage: the clean majority flows
untouched; humans only ever see the ambiguous slice.
Five techniques, one pipeline. Each one you already learned in isolation — assembled, it scales to a
volume no human could read.
This is CCA-F Scenario 6 — a Structured Data Extraction Pipeline, covering exam Domains 4 (Prompts/
Structured Output) and 5 (Reliability). It's the cleanest demonstration of how the Prompts plane and
the Reliability plane work together on a real throughput problem.
Next: you've built it all. Time to step back and see the whole stack — the finale.
We crack open every bone.
▶ Chapters
0:00 Messy invoices in, clean trustworthy rows out — at scale
0:45 The pipeline shape: raw doc → extract → validate → route → DB
1:30 Stage 1: forced-tool schema, the model MUST return the shape (04-01)
2:25 Few-shot locks EU conventions (decimal commas, DD/MM dates) (04-02)
3:20 Validate + retry with a max-1 ceiling (a bad doc can't burn 20 calls) (04-01)
4:15 Confidence routing: the 0.70 line splits auto-accept from human (04-05)
5:10 Cached case-facts: pin the vendor rules, ~90% cheaper on repeat (06-01)
6:05 Run 1,000 docs: ~92% auto-accept, ~8% to humans
7:00 The whole board: five techniques, one pipeline
7:50 CCA-F Scenario 6 — Domains 4 and 5
▶ Mentioned in this episode
• Forced-tool schema + validation-retry, max-1 ceiling (episode 04-01)
• Few-shot examples to lock conventions (episode 04-02)
• Confidence routing at ~0.7 (episode 04-05)
• Cached pinned case-facts, ~90% cheaper on repeat (episode 06-01)
• CCA-F Scenario 6 — Structured Data Extraction Pipeline (Domains 4/5)
▶ A precision worth flagging up front
The numbers on screen — the 0.70 confidence threshold, the ~92%/~8% auto-accept split, "~90% cheaper"
on cached reads — are calibration knobs and illustrative figures, not constants. Your confidence
threshold depends on your tolerance for a wrong row reaching the database; your auto-accept rate
depends on your document quality; and "~90% cheaper" is the cached-read ceiling, with a real floor
caveat we kept honest back in 06-01. Tune the thresholds to your cost-of-error, don't copy 0.70 because
a video used it.
#Claude #Anthropic #DataExtraction #StructuredOutput #ForcedTools #PromptCaching #CCAF #ClaudeAPI