Anthropic Blocked 1 Agent Action in 47,000. The Number Worth Copying Is 100%
By Surya Pratap
September 19, 2026
11 min read
AI & Technology
September 19, 2026
11 min read
One billion in, fifty outHover to exploreYesterday's article ended on a complaint: you cannot review what you do not log. Today there is a document that shows what the other end of that argument looks like when someone has actually built it, at a scale nobody else has published.
Figures in this article come from Anthropic's "Measurements for understanding the pace of AI development inside frontier labs", published 17 September 2026, and from reporting by Engadget, Quartz and FourWeekMBA. Every number is Anthropic measuring itself, which is a caveat Anthropic states plainly and I repeat in section 7. The reading of them, and everything from section 4 onward, is mine.
Anthropic published three measurements of its own AI research and development, proposed as things frontier labs should report routinely rather than as a one-off disclosure.
How much of the company's AI research work Claude leads, rated against an autonomy scale from AL0 to AL5 originally developed by Epoch AI.
How many internal agents run, how much of what they do is monitored, and what happens when a monitor objects.
What share of AI R&D compute went to safety work, measured over a single week.
The headline everyone picked up was the first one: Claude leads 26% of Anthropic's AI R&D work as of August 2026, up from under 1% in February. That number is being quoted without its second half, which changes it — more on that in section 5.
The second measurement is the one I would actually take to a build meeting.
As of August 2026, on Anthropic's primary internal platform:
A billion decisions at one end, fifty human reviews at the other, and a stated path between them.
Whatever else is arguable here, that funnel is a real engineering artefact. It is the first time a lab has published the arithmetic of how automated work gets narrowed down to something a human can actually attend to.
0.002% is the figure that will end up in slide decks, and it is close to uninformative on its own. Anthropic says so itself, in as many words: a rate that low is consistent with agents rarely attempting anything that warrants blocking, and equally consistent with a monitor that intervenes rarely by design.
Those are very different worlds, and no block rate can tell them apart. A monitor that blocks nothing scores identically to a fleet of impeccably behaved agents.
How a block rate fails as a metric
Push the logic to its ends. A monitor with no rules blocks 0% and looks perfect. A monitor that blocks everything blocks 100% and looks alarming while being perfectly safe and perfectly useless. The metric moves for reasons that have nothing to do with whether oversight is working, which means it cannot be used as a target — and the moment anyone does target it, it stops measuring even what little it measured.
The figures that do carry information are the ones about coverage:
"100% of actions pass through the monitor before execution" is a statement about architecture. It is verifiable, it is falsifiable, it does not move around with model behaviour, and it is the same claim whether your agents make a billion decisions a month or four hundred. "100% ingested afterwards" is the same kind of claim about the second layer.
If you copy one thing from this publication, copy the coverage commitment. Do not copy the target.
The instinct with a number like 30,000 agents is that none of this applies to a team of five. The structure applies; only the constants change.
| Anthropic's layer | What it does | The MVP-scale version |
|---|---|---|
| Online monitor, 100% pre-execution | Cheap, synchronous, blocks before the action lands | A policy check in the tool-call path: allowlist, argument validation, spend and blast-radius caps |
| Blocked actions to human review within a week | Bounded latency on the exception path | A queue with an owner and a rule for how stale an item may get |
| Offline monitor, 100% post-hoc | Reads everything, no latency budget, catches what the gate was not written for | A nightly pass over yesterday's transcripts with a cheap classifier |
| ~1–2 flags per thousand, ~50 to a human | Tuned so the human end is a number a human can do | Tune the threshold to the review capacity you actually have |
The tuning insight hiding in the numbers
Work backwards from the last column, not the first. Anthropic's offline flag rate lands at roughly fifty items a week reaching a person — a number one team can genuinely work through. The threshold was almost certainly chosen to produce that, rather than derived from first principles and then discovered to be manageable. Your threshold should be set the same way: decide how many transcripts a human will read each week, then set the bar wherever it produces that many.
The two-layer split is the part most small teams get wrong. They build one monitor, put it inline, and then have to keep it cheap and fast — which means it can only check things that are cheap and fast to check. The online and offline layers exist precisely because those constraints are different. The gate stops the obvious and the irreversible. The post-hoc pass is where you can afford to be slow, thorough and suspicious.
Back to the headline, because the way it is being quoted matters for anyone planning headcount against it.
The 26% figure is the share of measured AI R&D tasks Claude leads at AL4, and AL4 is defined as completing most of a task end-to-end from a high-level prompt while a human supervises. AL5 — no human in the loop — is 0%. Above 90% of measured tasks sit at AL3 or higher, where AI does large chunks of work under close human direction.
So the honest summary of the index is not "a quarter of the research is automated". It is that a quarter of the research is led by a model with a person still watching, and that the fully unsupervised category is empty.
And one methodology figure that deserves more attention than the 26%:
The ratings are produced by a model judge, and Anthropic reports that the judge and human raters agreed exactly 59% of the time, landing within one level of each other 97% of the time.
That is a genuinely useful disclosure, and it generalises straight to your evaluation set. A judge that agrees with a human three times in five is close to useless for gating a single task, and perfectly serviceable for tracking an aggregate over time — which is exactly what this index is used for. If you are using a model to score your own agent, that distinction is the whole game: aggregates tolerate noisy judges, individual pass/fail decisions do not.
Four things, in the order I would do them, sized for a team that does not have thirty thousand agents.
Write down your coverage number. Not the target — the actual fraction of agent actions that pass a check before execution. If it is not 100%, the gap is the list of things your agent can do that nobody is looking at.
Split the gate from the audit. One synchronous check on the tool-call path, cheap enough to run on everything. One asynchronous pass over the transcripts, slow enough to be thorough. Trying to get both from one component is what produces a monitor that does neither well.
Set the escalation threshold from your review capacity. Decide the number of transcripts a human will read each week first, then tune the classifier until that is what arrives. A queue nobody drains is not oversight.
Put a latency bound on the exception path. Anthropic's is one week for a blocked action. Yours can be whatever you can keep, but it needs to be written down, because an unbounded review queue quietly becomes a deny-list nobody revisits.
None of this requires a safety team. It requires deciding that the monitoring path is product, not an afterthought — and the reason to decide it now is that retrofitting coverage onto an agent that already has fifty tools is substantially harder than building it with five.
Every one of these numbers is Anthropic measuring Anthropic. No external party has verified any of the ratings. Anthropic says it plans to embed independent third-party evaluators with access comparable to internal risk teams; that is a stated plan, not an existing fact, and should be read as one.
The safety compute figures cover one week. 6% of AI R&D compute to safety, 12% within AI-driven AI R&D specifically, measured 13–20 July 2026. Anthropic is explicit that a single seven-day window is enough to show the measurement can be made and not enough to show a trend. I have left those numbers out of the argument above for that reason.
The automation index measures a frozen basket of task types. It tracks how automated known work has become, not whether new kinds of work are appearing — which is the thing most likely to move the real number.
"Monitored" is not "understood". Anthropic notes it has been doing this kind of monitoring for months rather than years, and that it is hard to be certain the picture of possible agent behaviours is complete. Full coverage of actions is not full coverage of failure modes.
The section 4 mapping is mine. Anthropic published its architecture, not a recommendation for five-person teams. The claim that the structure scales down is my inference, and the constants in that table are illustrative.
The interesting thing about this publication is not that the numbers are flattering — they are, and they are self-reported, and both of those should be discounted.
The interesting thing is that a lab published the shape of agent oversight at a scale where the shape is forced to be real. A billion decisions cannot be reviewed by good intentions. Something has to screen everything cheaply, something else has to read everything slowly, and the funnel between them has to terminate in a number a human being can actually work through on a Thursday afternoon.
That structure is not a frontier-lab luxury. It is what oversight looks like as soon as an agent takes more actions per day than you can read, which for most teams happens in about the third week.
Do not benchmark yourself against the block rate. Ask what fraction of your agent's actions pass a check before they happen, and be honest about the answer.
Sources: Anthropic, "Measurements for understanding the pace of AI development inside frontier labs", published 17 September 2026 — the AL0–AL5 scale, the 26% AL4 figure, the above-90% AL3 figure, the ~30,000 agent count, the 100% online and offline coverage claims, the 0.002% block rate and one-in-47,000 framing, the one-to-two-per-thousand offline flag rate, the ~100,000 transcripts a week and ~50 human escalations, the one-week review bound, the 6% and 12% safety compute figures for 13–20 July 2026, the 59% exact and 97% within-one-level judge agreement, and every caveat in section 7 are all as published there. The AL0–AL5 scale originates with Epoch AI. Engadget and Quartz, 18 September 2026, for the reporting and framing; FourWeekMBA for the publication date and a consolidated table of the figures. The mapping to small-team architecture in section 4, the argument about block rates in section 3, and the reading of the judge agreement figure in section 5 are mine. For the article this one follows — the agent input nobody logs — see your agent writes its own context. For the case that an audit trail can no longer come from the model explaining itself, see monitorability went the other way.
IdeaToMVP Academy
4-week live cohort for founders. Learn to ship AI agents, scope MVPs, and automate your business — taught by the same team that writes these guides.
