Can AI agents do senior expert work? What the evidence says
Can AI agents do senior expert work? What the evidence says
Five recent studies suggest AI can match or assist expert work on some tasks under some conditions, but the evidence is limited: GDPval had experts blind-grade AI output against human work on one-shot tasks, while a BCG field experiment found consultants with AI did 12.2% more tasks and worked 25.1% faster inside the "jagged technological frontier" but were about 19 points less likely to be correct outside it. A METR study found experienced developers took 19% longer with early-2025 AI tools despite forecasting a 24% speed-up, and studies of 150 autonomous agents on one dataset and of ICU prediction agents found wide variation in method choices and that single agents outperformed naive multi-agent systems.
Source: Hacker News AI · estimandlab.com