📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems have achieved near-saturation in core engineering skills for AI research, automating much of the engineering process. However, research activities are still less automated, leaving a residual gap. This shift could reshape AI development timelines and institutional strategies.
Recent developments in AI capabilities indicate that AI systems can now automate the majority of AI engineering tasks, with some benchmarks approaching full saturation. Meanwhile, automation of AI research activities remains less advanced, leaving a residual gap that could influence future AI development strategies. This shift is confirmed by multiple benchmark trajectories and recent research advances, marking a significant turning point in AI R&D.
Multiple independent benchmarks measuring AI capabilities in core AI research skills show rapid progress toward saturation. For example, the CORE-Bench, which tests research reproduction, has improved from 21.5% in September 2024 to 95.5% in December 2025, with the benchmark’s author declaring it ‘solved.’ This indicates AI systems can now handle complex research reproduction tasks at a level comparable to competent post-docs, effectively making research replication an engineering problem.
Similarly, the MLE-Bench, assessing performance on Kaggle competitions across NLP, vision, and signal processing, has seen scores rise from 16.9% in October 2024 to 64.4% in February 2026. This achievement places AI performance at a mid-tier human level, with the leaderboard paused to develop more accurate measurement methods, underscoring the rapid pace of capability growth. Additionally, advances in kernel design—such as automated GPU kernel generation and optimization—are increasingly moving from research papers to production-ready tools, further indicating that engineering tasks are becoming fully automatable.
Clark’s analysis suggests that the pattern across these benchmarks is consistent: AI capabilities are approaching saturation in core engineering skills, while research activities, which often involve higher-level creativity and hypothesis generation, are less advanced in automation. The structural implication is that research may itself be a form of large-scale engineering, which could accelerate the overall pace of AI development and reduce the residual gap.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
1000 AI Tools Directory 2026: The Ultimate Guide to AI Tools for Business, Productivity, Content Creation, Marketing, Coding, Design, Research and Automation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
GPU-Accelerated Computing with Python 3 and CUDA: From low-level kernels to real-world applications in scientific computing and machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
AI Automation Handbook: Building Smart Systems with Agents and MCP: A Practical Guide for Beginners
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
AI research reproduction benchmarks
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Near-Complete Automation of AI Engineering
The near-saturation of AI in engineering tasks signifies a potential paradigm shift in AI R&D, where the bottleneck shifts from engineering to the more complex, less automatable aspects of research. This could drastically reduce development timelines, lower costs, and democratize access to advanced AI capabilities. However, the residual challenge of automating research activities remains, raising questions about the future role of human creativity and oversight in AI innovation.
Institutionally, organizations may need to rethink their R&D strategies, investing more in managing AI-generated research and less in traditional engineering workflows. The possibility that research itself could become more automated suggests a future where AI-driven innovation accelerates, but also where the nature of scientific discovery may fundamentally evolve.
Recent Advances in AI R&D Capabilities
Over the past 16-21 months, multiple benchmarks have demonstrated rapid improvements in AI’s ability to perform core research and engineering tasks. The CORE-Bench, measuring research reproduction, has seen a 4.4× improvement, with the latest results indicating near-complete automation. The MLE-Bench, evaluating AI on Kaggle competitions, has shown a similar trajectory, reaching competitive human-level performance and prompting a pause for more accurate measurement. Advances in kernel design, including automated GPU kernel generation, further illustrate AI’s move toward production-grade engineering capabilities.
This pattern aligns with the broader ‘cascade’ of capability improvements across different domains, suggesting that AI’s engineering skills are approaching full automation. The remaining challenge is understanding how much of the research process—encompassing hypothesis generation, experimental design, and interpretation—can be similarly automated, a question still under investigation.
“The pattern across all benchmarks indicates that AI is nearing saturation in core engineering skills, which could fundamentally alter the structure of AI research and development.”
— Thorsten Meyer
Remaining Uncertainties About Research Automation
While engineering tasks are approaching full automation, it remains unclear how much of the research process—such as hypothesis formulation, experimental design, and interpretation—can be automated. The structural question Clark leaves open is whether research itself is a form of large-scale engineering, which could accelerate residual automation. The pace of progress suggests this gap may close faster than initially expected, but definitive evidence is still lacking.
Next Steps for Monitoring AI R&D Capabilities
Researchers and institutions will likely focus on developing benchmarks and tools to better measure AI’s research automation capabilities. Observing whether the trend toward automation continues across more complex, creative aspects of research will be critical. Additionally, organizations may need to adapt their R&D strategies to leverage AI’s engineering strengths while addressing the remaining challenges in automating scientific discovery.
Further empirical studies over the next 12-24 months will clarify how quickly AI can fully automate the research process, potentially leading to a new era of accelerated innovation.
Key Questions
What does it mean that AI can automate engineering tasks?
It means AI systems can now handle tasks like reproducing research experiments, optimizing code kernels, and managing infrastructure, which were previously done by human engineers.
Why is automating research still a challenge?
Research involves creative hypothesis generation, experimental design, and interpretation, which are less straightforward to automate than engineering tasks.
How might this shift impact AI development timelines?
If research automation progresses as engineering has, it could significantly shorten the time from idea to deployment, accelerating AI innovation cycles.
Are there risks associated with automating research activities?
Potential risks include reduced human oversight, challenges in validating AI-generated hypotheses, and ethical considerations in automated scientific discovery.
Source: ThorstenMeyerAI.com
