AI Agents Are Ready — We're the Ones Who Aren't

Opening

Reader, there’s probably not a single day these days when you don’t hear the phrase “AI agent.” AI that writes code for you, sorts your emails, even executes financial transactions. But almost no research has actually measured how much work people are handing over to these agents in practice.

In February 2026, Anthropic published a study analyzing millions of human-agent interactions across its coding agent Claude Code and its public API. The title is “Measuring AI Agent Autonomy in Practice.”

What makes this study interesting is that it doesn’t measure “how smart AI has become,” but “how much people are actually delegating to it.” And the results are quite telling. Today I want to use this study as a jumping-off point to talk about where the real bottleneck sits in the age of AI agents.

What 45 Minutes Means: Autonomous Working Time Has Doubled

Let’s start with the core data from Anthropic’s study. They measured how long Claude Code works on its own without human intervention — what’s called “turn duration.”

Most turns are short. The median is about 45 seconds, and that number has barely moved over several months. As new users keep flowing in, the overall average has stayed stable.

The really interesting part is the tail data. The longest sessions — the top 0.1% of turn durations — nearly doubled, from under 25 minutes to more than 45 minutes, between October 2025 and January 2026. In just three months.

What’s worth noting here is that this increase happened gradually, independent of new model releases. If autonomy were purely a function of model capability, you’d expect sharp jumps every time a new model shipped. But the graph traces a smooth upward curve instead. This suggests power users grew increasingly comfortable trusting the tool over time, and began handing it progressively more ambitious tasks.

Anthropic calls this phenomenon “deployment overhang” — the idea that the autonomy a model can actually handle is far greater than the autonomy it’s actually given in the real world.

Trust Accumulates: Experience Reshapes How We Supervise

The patterns in how users grant autonomy to agents are also fascinating.

New Claude Code users (fewer than 50 sessions) use “auto-approve”¹​ mode for only about 20% of their tasks. But experienced users with 750+ sessions push that figure above 40%. As experience builds, they gradually hand over more discretion to the AI.

But here’s where a paradoxical data point emerges. Experienced users use auto-approve more often — and at the same time, interrupt the AI mid-task more often too. New users interrupt at a rate of about 5% per turn, while experienced users climb to about 9%.

It looks contradictory, but it actually reveals a shift in supervision strategy. New users approve every single step — a “pre-approval” approach. So there’s little need to interrupt mid-task. Experienced users, by contrast, switch to a “monitoring” approach: let the AI run autonomously and step in only when something goes wrong. The rise in interruptions isn’t neglect — it’s evidence of active vigilance.

This mirrors exactly how we manage new hires versus veterans at work. You check in on every step with a rookie, but with a trusted team member you say, “Handle it, and let me know if you get stuck.” The same structure appears to be operating in the relationship between humans and AI.

The AI Raises Its Hand First: Agents That Stop Themselves

Humans aren’t the only ones supervising the agent. Claude stops itself too.

On the most complex tasks, the rate at which Claude Code asks the user for confirmation is more than double the rate at which users interrupt Claude. For the highest-difficulty tasks, Claude’s rate of voluntarily asking questions is 16.4%, versus a human interruption rate of just 7.1%.

Analyzing why Claude stops itself reveals this pattern:

• To present a choice of approaches (35%)

• To gather diagnostic information or test results (21%)

• To clarify an ambiguous or incomplete request (13%)

• To request credentials or access permissions (12%)

• To get approval before taking action (11%) Meanwhile, the reasons humans interrupt Claude are:

• Providing missing technical context or corrections (32%)

• Claude is being slow or overdoing the work (17%)

• Having received enough help to proceed alone (7%) This is an important safety property. It shows that an agent’s ability to recognize its own uncertainty and apply the brakes when unsure matters just as much as external safeguards. Anthropic is deliberately reinforcing this trait during model training, and is recommending the same approach to other model developers.

Beyond Coding: Where Agents Are Headed

Looking at the current landscape of AI agent usage, it’s overwhelmingly concentrated in software engineering. About 49.7% of all tool calls on Anthropic’s public API are coding-related. Next comes back-office automation (9.1%), marketing (4.4%), sales/CRM (4.3%), finance and accounting (4.0%), and data analysis (3.5%).

80% of all tool calls have some safeguard in place — permission restrictions or human approval requirements — and 73% involve some form of human oversight. Irreversible actions (like sending an email) account for only 0.8% of the total.

That said, there are frontier cases too. A small cluster of high-risk, high-autonomy activity exists — automated financial trade execution, accessing patient medical records, elevating security privileges. Anthropic believes many of these are likely security evaluations (red-team exercises), but acknowledges it can’t distinguish these from actual production use.

The reason software engineering adopted agents first is structural. Code can be executed to check whether it’s correct. You can run tests, compare results, and review before deployment. In fields like law, medicine, and finance, verifying an agent’s output requires expert knowledge in itself. So trust inevitably builds more slowly there.

MCP vs. CLI: The Technical Debate Over an Agent’s “Hands and Feet”

Let’s get a bit technical here. There’s an interesting debate underway about how AI agents interact with the outside world — specifically, the interface used to connect tools.

MCP (Model Context Protocol)²​ is an open standard Anthropic released in November 2024. It’s a protocol that standardizes how AI agents communicate with external tools, databases, and APIs. Think of it like a USB-C port — a “universal adapter” that lets you plug in once and use any tool. Within a year of launch, OpenAI, Google, and Microsoft had all adopted it, and by December 2025 it had moved under the Linux Foundation — becoming an industry standard remarkably fast.

But lately, working developers have started saying “CLI beats MCP.”

The problem is context efficiency. A typical MCP server loads the schema³​ of every available tool into the agent’s context window all at once. For example, a single GitHub MCP server exposing 93 tools consumes about 55,000 tokens just for the tool definitions. Add a database, Jira, and Microsoft Graph on top of that, and tool definitions alone can eat up over 150,000 tokens — leaving less room for the reasoning that actually matters.

CLI (command-line interface)⁴​ tools, by contrast, use existing commands like gh, az, and git directly. Because AI models have already learned how to use these tools from their training data, they can accomplish the same task using just commands and results — under 200 tokens — with no separate schema needed.

One engineer compared the two approaches on the task “extract a list of non-compliant devices and export to CSV.” The MCP approach ran out of context after 3-4 reasoning steps and had to split into separate sessions, while the CLI approach built the whole pipeline — including edge cases — in a single session.

Of course, this isn’t a strict either/or. MCP’s strength lies in standardization and ecosystem — thousands of servers, cross-platform support, and security governance are already built out. Anthropic itself recognizes this problem and has proposed a code-execution-based approach to MCP interaction: instead of loading every tool definition at once, the agent searches for and loads only the tools it needs, on demand⁵​, via code.

To me, this looks like the maturation of agent architecture. The early approach was “show all the tools at once,” but running into the physical constraint of context windows in practice has pushed things toward a more efficiency-oriented pattern of “only the tools you need, when you need them.” MCP itself isn’t disappearing — rather, smarter tool-discovery mechanisms are being layered on top of it.

Oz’s Lens

What struck me most reading this study was the concept of “deployment overhang” — the phenomenon where AI’s capability is already sufficient, but real-world usage lags behind it. This pattern keeps recurring across the tech industry. It happened with cloud computing, it’s happening with self-driving cars, and now it’s happening here too. The moment a technology’s bottleneck shifts from “performance” to “trust,” what determines the slope of the growth curve is no longer engineers — it’s organizational culture and governance⁶​.

What’s especially notable is that Anthropic explicitly stated that “regulation requiring pre-approval for every action isn’t appropriate.” The “monitor plus selective intervention” approach that experienced users naturally developed turns out to be more effective. That has real implications for regulatory debates.

And as someone who works with data, I want to flag this study’s limitations. Anthropic can only analyze its own product data. On the public API, they can only see individual tool calls — they can’t reconstruct full agent sessions. Claude Code data skews heavily toward software engineering. And the risk and autonomy scores were classified by Claude itself, which creates a structural limit on verification. This study is clearly a meaningful first step, but it’s worth remembering that the full picture is still just one piece of the puzzle.

To me, the question that really matters is this: “Who decides an AI agent’s autonomy?” And this study’s answer is clear — it’s not the model alone, nor the user alone, nor the product design alone. All three shape it together. Anthropic calls this “co-constructed autonomy,” and I think this framing will become a starting point for AI governance debates going forward.

Closing

Here’s the summary. AI agents’ autonomous working time nearly doubled in just three months — but what drove that increase wasn’t model upgrades, it was users accumulating trust. And an agent’s ability to stop itself when uncertain is functioning as a supervisory mechanism every bit as important as external safeguards.

If you want to dig deeper into this topic, I’d recommend starting with the “Experienced users” section of the original study — the part where trust and supervision play out in the data is especially striking. Next time, I’ll talk about how this expanding autonomy is affecting the labor market, drawing on another study Anthropic recently published.

References & Further Reading

• METR, “Task-Completion Time Horizons of Frontier AI Models”, 2026. — The industry-standard benchmark for measuring AI agent capability. Read alongside Anthropic’s study, it makes the gap between “capability” and “real-world use” even clearer.
• Anthropic Engineering, “Code execution with MCP: Building more efficient agents”, 2025. — Anthropic’s official engineering blog post covering MCP’s context-efficiency problem and its code-execution-based solution.
• Jannis Reinhard, “Why CLI Tools Are Beating MCP for AI Agents”, 2026. — A well-organized practitioner’s take on the MCP vs. CLI debate, with concrete token-cost comparisons.
• MIT Technology Review, “This is the most misunderstood graph in AI”, 2026. — An article untangling the misunderstandings and context around the METR graph, useful for understanding what “time horizon” actually measures.
• Feng, McDonald, Zhang, “Levels of Autonomy for AI Agents”, Knight First Amendment Institute, 2025. — A framework classifying AI agent autonomy into five levels, with a practical, user-role-centered taxonomy.

The author, Kwangseob Ahn, is a professor of business administration at Sejong University and lead consultant at OBF (Oswarld Boutique Consulting Firm). He teaches statistics and data analysis — business data management and business analytics — while leading GTM and AI strategy consulting in the field, designing the seam between technology and business. He has published academic research on a memory architecture for AI dialogue systems (HEMA) and runs Daily Arxiv, a daily curation of global AI papers. He holds a master’s from Korea University’s Graduate School of Technology Management and a KMBA. He is the author of Homo Brainless: The People Who Outsource Their Thinking.

Footnotes

1. Auto-approve: A setting in Claude Code that allows the AI to automatically carry out each action (editing files, running commands, etc.) without the user confirming it individually. It’s off by default, so manual approval is normally required every time. ↩

2. MCP (Model Context Protocol): An open protocol that standardizes how AI models communicate with external tools and data sources. Think of it as a “universal connector” for AI — much like USB-C lets you plug many different devices into a single port. ↩

3. Schema: A specification that defines the structure of data or a tool — essentially a manual describing “what inputs this tool needs, and what output it produces.” ↩

4. CLI (Command Line Interface): An interface for operating a computer by typing text commands. Unlike a GUI, where you click with a mouse, this means typing commands directly — like git push. ↩

5. On-demand: Fetching only what’s needed, only when it’s needed. Like Netflix streaming only the movie you want to watch, this means the agent loads only the tools it needs, exactly when it needs them. ↩

6. Governance: The decision-making structure and accountability system of an organization or system. AI governance refers to the rules and processes around “what AI is allowed to do, and who is responsible for it.” ↩