🔍 Read the full analysis: AI’s Output Is Abundant; Careful Review Isn’t on ThorstenMeyerAI.com
Get the latest gadgets delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
TL;DR
A source essay argues that AI has made it cheaper to produce mathematical manuscripts, software changes and contract drafts, but has not made expert review equally fast. The figures it cites point to a growing review bottleneck, though some software metrics come from companies that sell review tools and should be read with that in mind.
A recent analysis argues that AI is making work cheaper to produce faster than it can be checked, drawing on OpenAI’s mathematical manuscripts, software-development metrics and a contract-workflow evaluation. The evidence suggests that review capacity is becoming a constraint, although the cited measurements come from different sources and are not directly comparable.
The analysis says OpenAI published 722 mathematical manuscripts this week, generated from work on about 4,000 problems and grouped into 372 families. It reports that the average result took about three hours of compute. Some results have been formally checked using Lean, a proof-assistant system. OpenAI cautioned that some results not formalized in Lean could have issues, according to the source material.
The author contrasts that volume with the response to an earlier result from the same programme: a proposed counterexample to an old Erdős conjecture that was carefully checked by five leading mathematicians. The comparison illustrates the central concern: computer systems can produce many candidate results, while expert attention to determine their correctness and significance remains limited. The analysis cites the phrase “verification abundance, adjudication scarcity” to describe the mismatch.
Software figures point in a similar direction, but require care. Faros AI reported that teams in high-AI-adoption periods merged 98% more pull requests, while review time increased 91%. LinearB, examining 8.1 million pull requests across 4,800 organisations, reported that AI-generated changes waited 4.6 times longer for review to begin and were accepted 32.7% of the time, compared with 84.4% for human-written changes. The source notes that several data providers sell code-review tools, so their findings should be treated as attributed measurements, not neutral industry-wide estimates.
The referee shortage: AI made doing cheap and checking expensive
OpenAI’s model produced a maths result in about three hours of compute. Verifying one earlier result took five of the world’s leading mathematicians. That ratio is the next decade of work: producing is cheap and abundant; trusting is slow, human and scarce.
Some Lean-checked; OpenAI warns unformalized ones “could have issues.” Verification abundance, adjudication scarcity.
Faros AI. LinearB (8.1M PRs): AI changes wait 4.6× longer, accepted 32.7% vs 84.4%.
Real progress. Someone still has to find the other 45% before the work can be used.
Machine output arrives without reasoning a reviewer can interrogate. It looks locally clean and gives no clue where it’s wrong.
A prover confirms the proof proves its statement; tests confirm what tests check. Neither confirms it’s what was needed.
Contracts are signed, designs stamped, papers defended. Responsibility is institutional — you can’t hold a model to it.
of AI-agent pull requests got no human review at all (EASE 2026). Zero-review merges up 31.3% (Faros).
of reviewers deliberately deprioritise AI changes (LinearB). Good machine work waits behind bad.
OpenAI chose which maths families were significant. When referees can’t keep up, the producer’s filter becomes the review.
Budget review hours next to model spend.
Provers, types, tests, policy engines.
Experts only where consequences are high.
Who profits from generation pays for checking.
Keep some production human for learners.
The first automation question was which jobs AI would do. The better one is which jobs AI makes more necessary: the ones that check, adjudicate and take responsibility. Expect a referee premium — senior engineers, auditors, specialist lawyers, reviewing scientists become the binding constraint on how much AI output anyone can use.Accountability — standing behind a result — may be the most durable form of human work there is.
Review Capacity Becomes the Constraint
If AI increases the supply of drafts, proofs and code without increasing the supply of people able to assess them, organisations may not be able to use all the output safely or effectively. The scarce resource could shift from making a first version to deciding whether that version is correct, relevant and ready to rely on.
The practical consequences vary. A mathematical proof may need specialists to test both its logic and the importance of the statement it proves. Software teams may face longer queues or accept changes with insufficient review. In contract work, a draft still needs a qualified person to catch a missed approval rule or an unsuitable clause. The source says OpenAI’s GPT-6 Astra, developed with contract-software company Ironclad, met 55% of evaluation criteria on average across 11 tasks. That is described as a large improvement over the prior model, but leaves criteria that still need attention before the output can be relied on.
The analysis also identifies a workforce concern: junior staff often develop the judgment needed for senior review by doing the underlying work themselves. If AI takes over too much drafting or coding, fewer workers may get the experience that later enables them to assess complex output. This is a risk raised by the analysis, not a demonstrated outcome in the cited figures.
As an affiliate, we earn on qualifying purchases.
Three Fields, One Mismatch
The source’s argument draws on three distinct settings. In mathematics, formal proof tools can check whether a proof establishes its stated theorem, but they do not decide whether the theorem is the right one to pursue or what it contributes. Human mathematicians still make those judgments. The source says the disputed earlier counterexample illustrates how a result can require scrutiny beyond producing a proof.
In software, automated tests can check specified behavior, but cannot establish that the tests capture every requirement. The source cites a peer-reviewed 2026 study reporting that 61% of AI-agent pull requests received no human review before being merged or closed. Faros also reported that merges with zero review rose 31.3% during high-adoption periods. These are separate findings with different methods; they should not be combined into a single estimate.
In professional work, a model can draft or classify documents, but people and institutions retain responsibility for decisions. The analysis points to contracts signed by people, engineering designs approved by licensed professionals and research authors expected to answer for their work. Its broader point is that checking includes accountability, not just finding technical errors.
As an affiliate, we earn on qualifying purchases.
How Much Review Is Missing?
The cited measurements do not establish a single, representative rate of AI review failure across industries. The sources use different samples, definitions and time periods, and the software metrics include reports from vendors with commercial interests in review tools. The direction may be consistent across these sources, but the size and prevalence of the bottleneck remain uncertain.
The source material also does not provide the full methods behind the mathematics and contract-workflow claims, or the detailed comparison with earlier models. It is not clear how much human review the 722 manuscripts received, how the 11 contract tasks were weighted, or whether the evaluation criteria reflect the risks of real legal work. Nor do the figures show whether added review time led to safer outcomes, slower delivery, or both.
Finally, it remains uncertain whether new tools, changes to team processes or different training practices can expand expert review capacity quickly enough. The analysis makes a forecast about the value of people who can approve and take responsibility for AI-supported work; it does not establish how wages or hiring will change.
mathematical proof assistant software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Measuring Review Alongside Output
The next useful evidence will show not only how much work AI systems generate, but how it is reviewed and what errors remain. That means clearer reporting on review rates, time spent checking, changes requested, defects found and outcomes after deployment, with methods and comparison periods disclosed.
For the mathematics programme, readers will need to see which results receive formal verification and what independent experts conclude about their correctness and contribution. For software and contracting, further studies could compare AI-assisted and conventional work under the same review standards, while distinguishing vendor reports from independent or peer-reviewed research.
Organisations adopting these systems will also need to track whether junior staff still gain enough hands-on experience to develop judgment. The available material does not establish a timetable for resolving the mismatch. For now, the central question is whether review processes can expand without treating automated output as reliable simply because it is plentiful.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is the main development described?
A recent analysis combines examples from mathematics, software and contract work to argue that AI-generated output is growing faster than expert review capacity. It is an analysis of reported developments, not a new controlled study of all three fields.
Did OpenAI publish 722 fully verified mathematical results?
The source says OpenAI published 722 mathematical manuscripts, with some results formally checked in Lean. OpenAI cautioned that some results without formalization could have issues; the source does not say that every manuscript was independently verified.
What do the software figures show?
Reports cited by the analysis describe more pull requests, longer waits for review, lower acceptance rates for AI-generated changes in one dataset and substantial numbers of agent pull requests without human review in a separate study. Their methods and samples differ, and some sources sell review tools.
Why can’t AI simply check AI output?
Automated checks can test defined properties, such as whether a proof follows formal rules or code passes specified tests. They may not establish whether the original question, test or requirement was the right one, and they do not take legal or professional responsibility for the result.
What remains uncertain?
The available figures do not establish how widespread the review bottleneck is across the economy, whether it causes worse outcomes overall, or how quickly institutions can add review capacity. The cited evidence also does not settle how AI adoption will affect the training of junior professionals.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
