Engineering Is Automated. Research Is the Residual.

📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.

TL;DR

AI systems are now capable of automating most engineering tasks in AI research, reaching near-saturation levels on key benchmarks. Research remains less automated, but evidence suggests the gap may close faster than expected, potentially transforming AI development timelines.

Recent advances in AI capability demonstrate that systems can now automate the core engineering tasks involved in AI research, with benchmarks approaching saturation levels. This development shifts the focus from engineering to research, raising questions about the future pace of AI innovation and the remaining human role.

Multiple independent benchmarks—CORE-Bench, MLE-Bench, and kernel design research—show AI systems have achieved near-complete automation of key engineering skills relevant to AI research. For example, CORE-Bench, which measures the ability to reproduce computational research papers, reached 95.5% success in December 2025, with the benchmark’s author declaring it ‘solved.’ Similarly, MLE-Bench, assessing performance in Kaggle competitions, hit 64.4% in February 2026, reaching a level comparable to mid-tier human practitioners. These patterns suggest that the bottleneck in AI research is shifting from engineering execution to the more cognitively complex research phase, which remains less automated. Experts like Thorsten Meyer interpret this as evidence that the residual research challenge may close faster than the engineering milestone, especially if research itself becomes a form of engineering at scale.

Engineering Is Automated. Research Is the Residual.
DISPATCH / MAY 2026 CLARK EXTENDED · AUTOMATED AI R&D · OUTSIDE READ 02
▲ The Outside Read 02 Engineering / Residual · May 2026
Six Skill Benchmarks · The 99% Perspiration Thesis · Outside Read 02

Engineering is automated.
Research is the residual.

Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.

Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.

99%
Perspiration
Automated
/
1%
Inspiration
Residual
Edison · 150 years on · still right
The structural read
AI is excellent at the 99% of AI R&D — engineering, optimization, kernel design, fine-tuning. The 1% inspiration may be a permanent moat. Or it may dissolve as inspiration is recognized as compressed perspiration.
52×
AI speedup · Mythos · Anthropic CPU task
vs 4× human in 4-8 hours · 13× faster than researchers
95.5%
CORE-Bench · declared “solved” Dec 2025
Up from 21.5% Sep 2024 · paper reproduction · saturated
6 of 6
Skill benchmarks converging on saturation
CORE · MLE · Kernel · PostTrain · CPU · Alignment
1 / 700
Erdos problems · “interesting” solutions
Inspiration data point · ambiguous reading
CPU SPEEDUP TASK 2.9× → 16.5× → 30× → 52× IN 11 MONTHS · 13× HUMAN BASELINE CORE-BENCH SOLVED 21.5% → 95.5% IN 15 MONTHS · BENCHMARK AUTHOR DECLARED IT COMPLETE MLE-BENCH PAUSED 16.9% → 64.4% · LEADERBOARD PAUSED APRIL 2026 FOR FAIR-COMPARISON REWORK POSTTRAINBENCH AI 25-28% VS HUMAN 51% · HALF HUMAN BASELINE · THE RECURSIVE TRIGGER RESIDUAL QUESTION ERDŐS 13/700 · 1 INTERESTING · MOVE 37 STILL UNREPLACED AFTER 10 YEARS ENGINEERING IS AUTOMATED RESEARCH IS THE RESIDUAL CPU SPEEDUP TASK 2.9× → 52× IN 11 MONTHS · 13× HUMAN BASELINE CORE-BENCH SOLVED 21.5% → 95.5% IN 15 MONTHS
The six skill benchmarks · all converging on saturation

Six skills. One trajectory.

Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.

The six skill benchmarks · trajectory data
Five of six saturated or paused; one (PostTrainBench) at half human baseline — the recursive trigger.
CORE-BenchResearch reproduction
21.5% Sep 2024 → 95.5% Dec 2025 (Opus 4.5). Benchmark author declared it “solved.” 15 months. 4.4× improvement. Research replication = solved engineering problem.
SOLVED
MLE-BenchKaggle competitions
16.9% Oct 2024 → 64.4% Feb 2026 (Gemini 3). 16 months. Leaderboard paused April 2026 pending fair-comparison rework. ~Bronze-medal-or-better on 2/3 of 75 Kaggle competitions.
PAUSED
Kernel designGPU optimization
No single benchmark. Multiple production papers across 2025-2026. Meta uses LLMs for Triton kernels in production. AscendCraft for Huawei. From research curiosity to deployment standard.
PRODUCTION
PostTrainBenchAI fine-tuning AI
Opus 4.6 / GPT-5.4 at 25-28% vs human 51%. AI currently at half human baseline. The recursive self-improvement trigger — leading indicator for AI exceeding human on training AI.
HALF-HUMAN
Anthropic CPULLM training speedup
2.9× May 2025 → 16.5× → 30× → 52× April 2026. 11 months. Human baseline: 4× in 4-8 hours. Mythos is 13× faster than a researcher on a full workday’s task.
13× HUMAN
Automated alignmentAnthropic proof-of-concept
Anthropic’s AI agents beat human-designed baseline on scalable oversight. Small-scale, not yet production. The most consequential benchmark — AI doing AI alignment research is the recursive concern.
PROOF-OF-CONCEPT
Engineering is automated. The question is whether research is residual.
The 1% inspiration question · creativity data points
The No-BS Guide to AI for Trading & Market Research: How to Use ChatGPT, Claude & AI Tools for Market Analysis, Stock Research & Data-Driven Trading ... ... Required (The No-BS AI Playbooks Book 3)

The No-BS Guide to AI for Trading & Market Research: How to Use ChatGPT, Claude & AI Tools for Market Analysis, Stock Research & Data-Driven Trading … … Required (The No-BS AI Playbooks Book 3)

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Three data points. Mixed signal.

Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.

The creativity data · three observations
Inspiration data isn’t dispositive; the next 12-24 months produce the empirical resolution.
▲ Move 37 · 2016
AlphaGo’s creative move
10 yrssince · no replacement
Canonical example of AI producing creative-feeling insight. 10 years on, Move 37 hasn’t been replaced by a comparably impressive flash of insight. Capability has risen dramatically; discovery moments haven’t.
Weakly bearish signal · per Clark
▲ Erdős Problems · 2025-26
Math team + Gemini
13 / 7001 “interesting”
Team attacked ~700 problems with Gemini. Got 13 solutions; 1 deemed “interesting” (Erdős-1051). Conservatively framed: “slightly non-trivial,” “somewhat broader,” “mild.” 0.14% rate of interesting insights from massive parallel exploration.
Ambiguous · low yield, real result
▲ Centaur Discovery · 2026
Real math proof
substantialGemini contribution
UBC/UNSW/Stanford/DeepMind paper with “very substantial input from Google Gemini and related tools.” Real proof, real publication. “Centaur” framing — human + AI together — not AI alone. Real research advance through partnership.
Yes-evidence · with caveat

The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.

What Clark doesn’t develop · five strategic dimensions
AI Simply Explained by a Software Engineer

AI Simply Explained by a Software Engineer

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Five dimensions Clark gestures at but leaves underdeveloped.

Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.

Five strategic dimensions Clark doesn’t develop
Each affects the institutional response calibration for the 32-month window.
01
The competitive lab dynamic
Each lab publishes capability data as competitive positioning. Labs that automate R&D pull ahead structurally — their next model is trained by AI agents more capable than competitors’. No lab can unilaterally slow down without losing the race. Coordination problem at scale.
COMPETITION
02
The interpretability gap
When AI does the R&D, humans understand less about how next models are made. Hyperparameters, training data composition, optimization decisions — all from AI agents. Interpretability of outputs assumes you know how the model was built. The assumption is slipping.
INTERPRETABILITY
03
The brain drain question
Senior researchers move up the abstraction stack. Entry-level apprenticeship through engineering schlep is closed. Same “missing generation” dynamic as software engineering. Remaining human AI talent concentrates at frontier labs with the agent infrastructure.
LABOR MARKET
04
The volume thesis · more shots on goal
If inspiration is volume-derived, more compute for R&D exploration = more rare discoveries. Compute capacity directly translates to research output velocity. Compute geography becomes research geography. Frontier labs with privileged compute capture the volume upside.
COMPUTE = RESEARCH
05
The recursive alignment concern
Automated alignment research means AI produces the alignment knowledge AI is aligned by. Verifier and system are the same generation of AI. Anthropic’s proof-of-concept makes this operational. Current peer review and publication frameworks weren’t designed for this.
VERIFIER-SUBJECT UNITY
The two readings · does inspiration bound the trajectory?
Evals for AI Engineers: Systematically Measuring and Improving AI Applications

Evals for AI Engineers: Systematically Measuring and Improving AI Applications

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Two readings. Different equilibria.

The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.

Two readings of the residual question
Both consistent with Clark’s evidence. The next 12-24 months resolve the empirical question.
▲ READING 01 · INSPIRATION IS BINDING
Research is qualitatively distinct.
Creative insight is something AI fundamentally lacks. Rare discovery moments don’t accelerate with capability. Research bounds the trajectory at human-research-pace.
Supporting evidence: Move 37 unreplaced for 10 years. Erdős discovery at 0.14% yield. PostTrainBench at half human baseline. Centaur configuration prevalent — AI not autonomous in research.
Consequence:
Productivity multiplier years
▲ READING 02 · INSPIRATION IS COMPRESSED PERSPIRATION
Research is engineering at scale.
Rare discovery moments are an artifact of low-volume exploration. More shots on goal yields more discoveries proportionally. Research dissolves as automated R&D scales.
Supporting evidence: CPU speedup at 13× human on optimization tasks. Six benchmarks converging on saturation. Vaswani et al. transformer insight emerged from iteration. Inspiration historically inseparable from perspiration.
Consequence:
Recursive loop operational
Stakeholder implications · five audiences
AI Robotic Arm Kit Hiwonder SO-ARM101 Embodied Imitation Learning Open Source 6-Axis Robot Arm 12 High-Torque Bus Servo Motors AI Vision Recognition (Advanced Kit, Included 3D Printed Part, Assembled)

AI Robotic Arm Kit Hiwonder SO-ARM101 Embodied Imitation Learning Open Source 6-Axis Robot Arm 12 High-Torque Bus Servo Motors AI Vision Recognition (Advanced Kit, Included 3D Printed Part, Assembled)

【End-to-End Imitation Learning】Hiwonder SO-ARM101 robot arm is an embodied intelligent hardware platform compatible with the Lerobot open-source framework….

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Five audiences. Asymmetric cost of being wrong.

The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.

Stakeholder implications · by audience
Career, research strategy, policy framework, investment thesis, public engagement.
▲ FOR AI RESEARCHERS
IN INDUSTRY
Senior-as-supervisor is the durable role.
Engineering work — kernel design, training optimization, paper reproduction — is being automated. Career value moves up the abstraction stack: research direction setting, supervision of AI agents, validation of AI-produced outputs. Plan for the supervisor role; treat the implementer role as table stakes.
▲ FOR AI RESEARCHERS
IN ACADEMIA
Inspiration-heavy work is the comparative advantage.
Academic labs can’t compete on volume with frontier-lab automated R&D pipelines. Focus on the inspiration-heavy work: theoretical foundations, interpretability methodology, alignment frameworks, evaluation design. 1 deep insight beats 1000 quick experiments in the bounded-academic-compute regime.
▲ FOR
POLICYMAKERS
The framework is built for human researchers.
Current policy treats AI R&D as something done by human researchers in regulated organizations. Framework breaks when AI agents do most of the R&D. Liability for AI-produced research outputs? Corporate disclosure for AI-driven research? Regulation when researcher and subject are both AI? None of these have current answers.
▲ FOR
INVESTORS
Lab competition is productivity multiplier #2.
(a) Labs with the best automated R&D pipelines pull ahead structurally. Anthropic CPU speedup (2.9× → 52×) is the publicly available signal. (b) Compute as research input — the volume thesis means compute capacity translates to research velocity. Compute supply governance is the new AI research moat.
▲ FOR
EVERYONE ELSE
The wedge has produced the recursive loop.
The coding singularity piece argued coding is the wedge into recursive self-improvement. This piece shows the wedge has produced the capability set required for the loop to be operational at the engineering layer. The residual question — research — resolves over the next 12-24 months. What gets built institutionally during that period determines the equilibrium.

Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.

— The structural read · May 2026

Implications for AI Development and Human Roles

The automation of core engineering tasks means AI systems can now handle much of the technical execution involved in research. This could accelerate AI development timelines significantly, reducing the need for human intervention in routine engineering work. However, the remaining research phase—conceptualization, hypothesis generation, and innovation—may still require human insight. If research also becomes automatable, the pace of AI progress could accelerate further, potentially leading to a new era of rapid innovation and raising questions about the future of human involvement in AI research and development.

Progress Patterns in AI Capabilities and Benchmark Saturation

Historically, AI development has been constrained by the difficulty of automating core engineering and research tasks. Recent data from 2024-2026 shows a rapid progression across multiple benchmarks. CORE-Bench, measuring research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025. MLE-Bench, evaluating Kaggle competition performance, rose from 16.9% in October 2024 to 64.4% in February 2026. The pattern across these benchmarks indicates a nearing saturation point, suggesting that current AI systems are approaching the limits of automation in engineering tasks. Meanwhile, research-focused capabilities are still developing but show promising signs of rapid progress, with continuous publication of new kernel design techniques and automation methods throughout 2025 and 2026.

“The pattern across multiple benchmarks shows AI nearing saturation in core engineering skills, shifting the residual challenge to research, which may close faster than expected.”

— Thorsten Meyer

Unresolved Questions About Research Automation Pace

While engineering tasks are nearing full automation, it remains unclear how quickly research itself can be fully automated. The structural question is whether research can be reduced to engineering at scale, which could accelerate progress further. The timeline for this transition is still uncertain, and the technological breakthroughs needed to automate higher-level research tasks are not yet confirmed.

Monitoring Progress in AI Research Automation

Researchers and industry observers will focus on the continued development of AI systems capable of automating research tasks, including hypothesis generation, experimental design, and innovation. Key milestones include further improvements in benchmarks, publication of new automation techniques, and potential breakthroughs in automating research-level cognition. The next 12-24 months will be critical for assessing whether research automation can match engineering progress, potentially reshaping the timeline of AI development.

Key Questions

What are the main benchmarks showing AI automation progress?

CORE-Bench measures research reproduction, MLE-Bench assesses Kaggle competition performance, and kernel design research evaluates automation in low-level infrastructure tasks. All three show rapid progress approaching saturation.

Does automation mean AI can now fully replace human researchers?

While core engineering tasks are highly automated, research involves complex cognition that remains less automated. The timeline for full automation of research is still uncertain.

What are the implications for AI development timelines?

If research automation accelerates as engineering has, the pace of AI progress could increase significantly, potentially leading to faster breakthroughs and shifts in industry leadership.

Are there risks associated with automating research?

Automating research could lead to rapid progress but also raises concerns about oversight, safety, and the potential for unintended consequences. These issues require careful attention as automation advances.

Source: ThorstenMeyerAI.com

Nothing in this article is financial or investment advice. Cryptocurrency and precious-metal investments carry significant risk — do your own research and consider a licensed advisor.
You May Also Like

The Delegation Ladder: The Four Agentic Loops, And What Each One Lets You Stop Doing

Exploring the four agentic loops in AI design, their functions, and implications for automation and control in AI systems.

Every Benchmark Launched 2023-2024 Has Fallen — The METR / SWE-Bench / CORE-Bench / MLE-Bench / PostTrainBench Sequence

Every major AI research benchmark launched between 2023 and 2024 has reached or is nearing saturation, indicating rapid advancements in AI capabilities.

Undervolting Your GPU for Local Inference: Lower Heat, Same Tokens/sec

Undervolting your GPU via power limiting can significantly lower heat and noise during AI inference without sacrificing tokens/sec, confirmed by recent tests.

Google will pay SpaceX $920M per month for compute

Google will pay SpaceX $920 million per month from October 2026 to June 2029 for access to approximately 110,000 GPUs and CPUs, as part of a major AI compute deal.