In a previous report last month, we discussed Mercor’s innovative benchmark designed to evaluate AI agents’ proficiency in demanding professional domains, such as legal and corporate analysis. At that time, the initial performance metrics were quite modest, with all leading laboratories achieving scores below 25%. This led us to conclude that legal professionals were, for the foreseeable future, secure from AI-driven job displacement.
However, the pace of AI advancement is remarkably swift, with capabilities often transforming significantly in just a matter of weeks.
Indeed, this week’s introduction of Opus 4.6 has dramatically altered the competitive landscape. Anthropic’s latest model demonstrated a performance just under 30% in single-attempt evaluations, and an impressive average of 45% when afforded additional opportunities to tackle complex problems. It is noteworthy that this release incorporated several advanced agentic functionalities, including "agent swarms," which likely played a crucial role in enhancing its multi-step problem-solving abilities.
This leap in scores represents a substantial improvement over prior state-of-the-art achievements and underscores the unrelenting progress in foundation model development. Brendan Foody, CEO of Mercor, expressed particular astonishment, stating, “jumping from 18.4% to 29.8% in a few months is insane.”
While a score of thirty percent remains a considerable distance from full proficiency at 100%, suggesting that legal professionals need not anticipate immediate replacement by machines, it certainly warrants a reassessment of the confidence they might have held just a month ago.
The Editorial Staff at AIChief is a team of professional content writers with extensive experience in AI and marketing. Founded in 2025, AIChief has quickly grown into the largest free AI resource hub in the industry.