Skip to content

Labs · EarningsLens

The small model was enough

EarningsLens reads US earnings releases into checked figures, each tied to the cell it came from, and grades itself against the company’s own XBRL. On 118 held-out releases, Claude Haiku at $5 per 1,000 releases scored the same as Sonnet at $105 and a router at $61: the differences were within run-to-run noise. So the engine serves Haiku.

  • 97.25%F1 on 1,730 held-out figures
  • $5.27per 1,000 releases, Claude Haiku at list price
  • 0numbers shown that failed the in-span check0 by construction: a number not printed in the cell it cites is shown as not found
  • 17 sp95 time to read a release

How did each setup score on releases it never saw?

4 ways to read a release, graded against the companies’ own XBRL. The prompt was tuned on other releases.

4 setups on 118 held-out releases, golden set v4
SetupF1PrecisionRecall$ per 1,000p95First passHow it ranResults file
router-signals claude-haiku-5-5 → claude-sonnet-5-5, 51% sent to the larger model97.28%99.45%95.20%$61.1937.2 s81%direct2026-11-21T134309Z-router-signals-test.json
haiku-low (served) claude-haiku-5-597.25%99.51%95.08%$5.2716.9 s78%direct2026-11-21T164319Z-haiku-low-test.json
haiku claude-haiku-5-597.22%99.51%95.03%$3.62not measured79%batch2026-11-21T161931Z-haiku-test.json
sonnet claude-sonnet-5-597.09%99.45%94.85%$104.91not measured84%batch2026-11-21T132920Z-sonnet-test-on-v4.json
  • Criterion 1: the router’s F1 is at least 97%: met (97.28%).
  • Criterion 2: the router costs less than Sonnet, at no more than one point less F1: met ($61.19 against $104.91 per 1,000, with 0.1 point more F1).

The golden set: 200 releases from 100 companies, 82 releases from 41 companies to tune the prompt on and 118 releases from 59 companies held out, scored on 1,730 figures. Every answer comes from the company’s own XBRL, never typed by hand.

Run-to-run noise is 0.17 point of F1, measured between 2 runs of one setup on the dev split. haiku-low’s own two runs on the held-out releases were 0.18 point apart: 97.25% directly and 97.07% as a batch. So differences of about that size aren’t a ranking.

Every cost is at list price. Runs marked batch went through Message Batches, which bills half and measures no latency, so their cost is doubled here to compare.

In a batch, 62% of haiku’s prompt was read from cache, against 24% in haiku-low’s direct run, so its $3.62 understates a live read.

router-signals is flagged as run with uncommitted changes. Its code was committed; the flag comes from the Sonnet results files, untracked when it ran (the runner counts untracked files).

How does a release become checked figures?

6 steps. Only the extraction and its retry call a model; code does the rest, and code decides what is shown.

  1. Find the release

    The engine finds the company’s 8-K filed under Item 2.02, results of operations, on SEC EDGAR.

  2. Parse it to cells

    It parses the release exhibit into paragraphs and table cells, each with an address such as t28/r6/c3, so every number can point at where it was printed.

  3. Extract with one call

    One structured-output call to Claude Haiku names, for each figure, the cell it came from and the number exactly as printed. Code, not the model, turns that into a value.

  4. Check every number

    Code checks that each number is printed in the cell it cites, that its scale matches the table’s note, that earnings per share agree with profit and shares, and that the period is the one the release covers.

  5. Retry once

    If a check fails, the model gets one retry that names the failed checks. A figure that still fails is shown as not found, never as a guess.

  6. Grade against the filing

    When the company files its 10-Q or 10-K, each figure the filing tags is graded against its XBRL: the company’s own numbers. Figures the filing doesn’t tag, and a fourth quarter’s per-share figures, aren’t scored.

What does one run look like?

A real extraction, recorded from the engine with its own tracing. Every step, in order.

NETFLIX INC (NFLX), Q3, read by haiku-low on 2026-11-21 at engine commit b2f29e0: 8.9 s in all, $0.0039 for 1 model call.

  1. Fetch and parse the release1.3 s
    fetch-and-parse

    timed by the recorder · 1 exhibit read from SEC EDGAR

  2. Lay it out for the model302 ms
    render-release

  3. Locate the figures7.3 s
    locate-figures

    claude-haiku-5-5 · 23,342 tokens in · 1,954 out · $0.0039

  4. Check each figure2 ms
    check-figures

    attempt 1 · 14 figures found · every check passed

  5. Grade against the filing0 ms
    grade

    graded against the 10-Q’s XBRL · 14 of 14 right · 0 missed · 0 wrong · 0 spurious

This recording passed every check on the first attempt. In haiku-low’s test run, this same release failed in_span on its first attempt and passed on the retry. Across that run, 22% of releases failed a check and retried.

See this release in the app, each number highlighted where it was printed.

What keeps a wrong number off the screen?

The model reads; code checks. A figure that can’t be checked against the release isn’t shown as a value.

  • Filing text is untrusted

    A release goes to the model as data, never as instructions. The most text in it can do is change an answer, and every answer is checked against the cell it cites.

  • The model has no tools

    It can’t browse, run code or call anything. It reads one release and answers in a fixed schema.

  • A number must be where it says

    A figure is shown only if its number is printed in the cell it cites. That check is code, and it runs on every answer.

  • “Not found” beats a guess

    A figure that fails its checks after the retry is shown as not found, with the checks it failed.

  • Visitors send a ticker, nothing else

    The app takes a ticker from SEC’s own list. No free text from a visitor ever reaches a model.

What did it get wrong?

Every miss of every setup, with the release and the filing that graded it, so you can check it yourself.

haiku-low: 92 misses (84 missed, 0 wrong value, 8 spurious)

F1 by field: cash 94.9%, diluted_shares 96.3%, eps_basic 95.0%, eps_diluted 97.6%, net_income 99.9%, operating_income 98.1%, revenue 98.1%, total_assets 90.8%

eps_basic: 17
diluted_shares: 16
revenue: 12
eps_diluted: 11
total_assets: 18
cash: 9
operating_income: 8
net_income: 1
router-signals: 90 misses (81 missed, 1 wrong value, 8 spurious)

F1 by field: cash 94.9%, diluted_shares 96.3%, eps_basic 95.0%, eps_diluted 98.0%, net_income 99.9%, operating_income 98.1%, revenue 98.1%, total_assets 90.4%

eps_basic: 17
diluted_shares: 16
revenue: 12
eps_diluted: 9
total_assets: 18
cash: 9
operating_income: 8
net_income: 1
haiku: 93 misses (85 missed, 0 wrong value, 8 spurious)

F1 by field: cash 94.9%, diluted_shares 96.3%, eps_basic 95.0%, eps_diluted 97.6%, net_income 99.9%, operating_income 98.1%, revenue 98.1%, total_assets 90.3%

eps_basic: 17
diluted_shares: 16
revenue: 12
eps_diluted: 11
total_assets: 19
cash: 9
operating_income: 8
net_income: 1
sonnet: 96 misses (87 missed, 1 wrong value, 8 spurious)

F1 by field: cash 94.3%, diluted_shares 96.3%, eps_basic 95.0%, eps_diluted 98.3%, net_income 99.0%, operating_income 98.1%, revenue 98.1%, total_assets 90.4%

eps_basic: 17
diluted_shares: 16
revenue: 12
eps_diluted: 8
total_assets: 18
cash: 10
operating_income: 8
net_income: 7

What does it cost?

Under a cent a release for haiku-low at list price, under caps that stop spending rather than bill more.

  • router-signals: $61.19 per 1,000 releases at list price; its test run was billed $7.22.
  • haiku-low: $5.27 per 1,000 releases at list price; its test run was billed $0.62.
  • haiku: $3.62 per 1,000 releases at list price; its test run was billed $0.21, at the batch’s half price.
  • sonnet: $104.91 per 1,000 releases at list price; its test run was billed $6.19, at the batch’s half price.

The 4 test runs on this page were billed $14.25 together. Recording the traced run cost $0.0039.

Live use is capped three ways: a meter that stops live reads at $10 a month, 5 live reads a day for each visitor, and Anthropic’s own spending limit on the workspace behind them. A release already read is served from the cache and costs nothing.

What’s next?

  1. Alerts on WhatsApp

    A message when a company you follow reports, with its checked figures (the notifications module).

  2. A regression gate in CI

    Every change to the prompt or the checks reruns the evals before it ships, once GitHub Actions runs again in December.