Agentic AI, Explained for the Mid-Market Leader

The Founders’ AItrained on 25 years of our work

Want this made concrete for your company? Ask me — or pick one:

What is agentic AI — and does my company actually need it? What should I do first? Take the 90-second readiness read

Ask

You have heard the word “agentic” more times this quarter than in your entire life before it. Every software vendor has bolted it onto their homepage. Every conference has a track. Every board deck now has a slide. And almost none of it comes with a definition a busy leader can actually use to make a decision.

So start with the number that should frame the whole conversation. In June 2025, Gartner predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027 — not because the technology failed, but because of “escalating costs, unclear business value, or inadequate risk controls.” A month earlier, MIT’s Project NANDA had put a sharper edge on it: across more than 300 enterprise AI initiatives, 95% showed no measurable return on the profit-and-loss statement. The researchers were explicit that the cause was not the models. It was an organizational “learning gap” — tools that don’t retain feedback or adapt to how the business actually works.

Hold those two facts next to the marketing and you get the real shape of 2026: agentic AI is simultaneously the most important shift in enterprise software in a decade and the category where most of the money is currently being wasted. Both things are true. And the gap between them is almost entirely about architecture and governance — the parts nobody is selling on a homepage.

This guide is the plain-language version a mid-market leader can act on: what agentic AI actually is, how to tell it apart from the chatbots and automation you already have, why so many projects stall, and what the honest first move is for a company that doesn’t have a research lab or a 200-person AI team. We build and run agentic systems for our own firm and for mid-market clients, so the frame here is operational, not theoretical.

The Word Is Everywhere. The Working Reality Is Narrow.

Those three numbers are not an argument against agentic AI. They are an argument for doing it deliberately. The category is real and moving fast — Gartner separately expects 40% of enterprise applications to include task-specific AI agents by the end of 2026, up from fewer than 5% in 2025. That is one of the steepest adoption curves enterprise software has ever recorded. But a steep adoption curve and a high failure rate at the same time tells you something specific: the technology works, and most organizations haven’t built the conditions for it to pay off.

There is also a definitional fog to cut through first, because the word has been stretched to mean almost anything. Gartner estimated that of the thousands of vendors claiming “agentic AI,” only around 130 offer genuine agentic capability — the rest is what the firm bluntly calls “agent washing”: chatbots, robotic process automation, and assistants relabeled for the moment. When the market itself can’t reliably tell agentic AI apart from its predecessors, a leader needs a working definition more than ever. So let’s build one.

What Agentic AI Actually Is

Strip away the marketing and agentic AI describes a specific behavior: software that pursues a goal across multiple steps, choosing its own actions along the way. IBM frames the operating loop in four moves — perceive, reason, act, learn — with a large language model acting as the orchestrator that decides what to do next based on what it just observed. The difference from the generative AI you already know is not intelligence; it’s autonomy. Generative AI produces content when you ask. Agentic AI decides what to do and does it.

The cleanest working distinction comes from Anthropic’s engineering team, and it is worth adopting verbatim because it cuts through most of the confusion. They separate workflows — “systems where LLMs and tools are orchestrated through predefined code paths” — from agents — “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.” A workflow runs on rails you laid down in advance. An agent decides the route. Both are “agentic systems,” and — this is the part most vendors skip — Anthropic’s own advice is to start with the simplest thing that works, a single model call or a fixed workflow, and escalate to full agency only when the task genuinely demands it.

That advice matters because the failure data suggests most organizations are doing the opposite: reaching for autonomous agents when a simple workflow would have been cheaper, more reliable, and easier to govern. The question a leader should ask a vendor is not “is it agentic?” It’s “does this task actually need an agent, or are you selling me autonomy I’ll have to babysit?”

The practical test for whether something is truly agentic comes down to three properties working together: it takes multiple steps toward a goal without being re-prompted at each one; it uses tools — it can search, call an API, write to a system, run code — rather than only producing text; and it adapts — if step three fails, it notices and tries something else. A model that only answers questions is generative. A script that runs the same fixed sequence every time is automation. Something that decides, acts, and adjusts across steps is an agent. Most of what gets sold as “agentic” fails at least one of those tests.

Agentic AI vs. Chatbots, Generative AI, and RPA

Because “agentic AI” is being pasted onto older technology, the fastest way to understand it is by contrast with the three things it’s most often confused with.

A chatbot is a tool you operate turn by turn. You ask, it answers, you ask again. It has no goal of its own and takes no action in your systems — it’s a conversation, not a colleague. The moment you close the tab, nothing continues.

Generative AI — the broad category chatbots belong to — produces content: text, code, images, summaries. It’s extraordinary at drafting and analysis, but it’s reactive. It generates when prompted and then waits. Agentic AI is built on top of generative models but adds the missing verbs: decide, act, check, continue.

Robotic process automation (RPA) is the one leaders most often mistake for agentic AI, because both “do things” in your systems. But RPA follows a fixed, brittle script — click here, copy this field, paste it there — and breaks the moment the screen changes. It has no judgment. An agent handles the same category of work but reasons about it: if the invoice is in an unexpected format, RPA fails and an agent adapts. The difference is the difference between a player piano and a musician.

That reframing is the whole point for a leader. You already know how to manage delegation, because you do it with people every day. You don’t hand a new hire the company credit card and root access on day one. You scope what they can do, you check their work until you trust it, and you keep accountability for the outcome. Agentic AI is the same management problem in a new medium — and the organizations that treat it as a management problem rather than a technology problem are the ones staying out of Gartner’s 40%.

Why Now: What Changed in 2024–2025

Agents aren’t a new idea — researchers have chased them for years. Three things converged recently to make them practically buildable, and understanding them helps you judge whether a vendor’s product is riding real capability or just the hype.

First, tool use became reliable. Modern models can decide to call an external function — a search, a database query, a calculation — and use the result, rather than hallucinating an answer. That single capability is what turns a text generator into something that can act in the world.

Second, a common standard emerged for connecting models to your systems. In late 2024, Anthropic open-sourced the Model Context Protocol (MCP), an open standard for connecting AI models to tools and data. Before it, every integration was a bespoke, brittle piece of engineering; MCP made “give the agent access to this system” a repeatable pattern rather than a custom project. It has since been adopted across the industry. This is the unglamorous plumbing that moved agents from demo to deployable.

Third, reasoning improved enough to plan. Newer models can break a goal into steps, sequence them, and correct course mid-task — the difference between a model that answers and a model that strategizes. How you assemble those steps — as a fixed sequence or a flexible loop — is itself an architectural decision with real consequences, one we dig into in loops versus graphs.

None of this made agents easy. It made them possible — which is exactly the moment a category fills with vendors claiming more than the technology can yet reliably deliver.

Why Most Agentic AI Projects Fail (And Why That’s Normal)

Here is the part the vendor decks leave out, and it’s the most useful part for a leader to internalize: the current failure rate is not an anomaly. It’s the expected state of a category this young, and knowing why projects fail tells you exactly what to build instead.

MIT’s NANDA researchers found that despite an estimated $30–40 billion in enterprise generative AI spending, only about 5% of integrated pilots were extracting measurable value. Their diagnosis was specific and it did not blame the models: the systems “do not retain feedback, adapt to context, or improve over time.” In other words, companies bought intelligence and forgot to build the memory, the context, and the feedback loops that would let that intelligence compound. They deployed a brilliant new hire and gave them no onboarding, no access to how the company actually works, and no manager. Of course it stalled.

McKinsey’s 2025 State of AI survey fills in the adoption picture: 88% of organizations now use AI in at least one function, but only about one-third have begun to scale it at the enterprise level, and just 6% qualify as “AI high performers” with more than 5% EBIT impact. Deloitte’s enterprise research adds the governance gap directly: close to three-quarters of companies plan to deploy agentic AI within two years, but only 21% report a mature governance model for agents. The intent is nearly universal. The readiness is rare.

The most instructive story in the category is Klarna’s. The fintech’s AI customer-service assistant, built on OpenAI, handled 2.3 million conversations in its first month, automated roughly two-thirds of the chat volume, cut average resolution time from eleven minutes to under two, and was projected to drive about $40 million in profit improvement. It was the poster child for agentic customer service. Then the company’s CEO publicly acknowledged they had cut human staff too aggressively — and began rehiring specialist human agents to handle the complex, ambiguous, emotionally charged cases the AI couldn’t. Klarna is not a story about AI failing. It’s a story about a company learning where the line between delegation and abdication actually sits — and having to walk back across it. That line is the whole game.

What Agentic AI Actually Means for a Mid-Market Company

If you run a company between 50 and 5,000 people, the honest framing is different from what you’ll hear at an enterprise conference. You don’t have a dedicated AI function, a governance office, or a research team. That sounds like a disadvantage. In two specific ways it’s an advantage — and in one specific way it’s a real risk.

The advantage is proximity. In a mid-market company, the person who understands the workflow, the person who can change it, and the person who owns the outcome are often in the same room — sometimes the same person. You can redesign a real process around an agent in weeks, without the change-committee drag that slows a 50,000-person enterprise to a crawl. The best early agentic use cases are unglamorous and internal: triaging inbound support, drafting and routing routine correspondence, reconciling data across systems, monitoring for exceptions and escalating the ones that matter. Work that is high-volume, rule-bounded, and expensive in human hours — where an imperfect result is cheap to catch and correct.

The risk is the mirror image of the advantage. McKinsey’s data shows agent adoption is concentrated in functions that already have strong technical governance — IT, engineering, knowledge management. A mid-market company deploying autonomous agents without that scaffolding is structurally the population most likely to land in Gartner’s 40% cancellation bucket, not least likely. Which leads to the most important sentence in this guide, and the one most vendors will never say to you:

Most mid-market companies should not deploy autonomous agents yet. They should build the architecture that makes agents safe and valuable first — and for many workloads, a well-designed workflow beats an agent on cost, reliability, and sleep-at-night.

That is not a counsel of caution for its own sake. It’s the pattern that separates the 6% of high performers from everyone else. The companies getting real value didn’t buy more agents. They built the conditions — shared context, structured instructions, governance, feedback — that let even a modest amount of AI compound. We wrote about that architecture-over-accumulation pattern in why more agents is making your business less intelligent, and about what the 20% who see returns actually built in the agentic organization ROI gap.

The Governance Question: Human In the Loop, or Above It?

The instinct when you hear “autonomous software acting in our systems” is to demand a human approve every action. That’s the right instinct on day one and the wrong operating model by day one hundred — and the data now shows why with unusual precision.

Anthropic studied how people actually supervise agents over time using its own coding tool. Users with fewer than 50 sessions let the agent act automatically about 20% of the time; by 750 sessions, more than 40% did. Trust grew — but gradually, and never into blind trust. The striking finding: experienced users intervened more often, not less, even as they granted more autonomy. They shifted from pre-approving every action to monitoring outcomes and stepping in when something looked wrong. Anthropic calls this “a shift in oversight strategy,” and it is the single most useful mental model a leader can take from the whole field.

We call the destination human above the loop rather than human in the loop. In the loop means a person approves each step — necessary early, but it doesn’t scale and it wastes the very leverage the agent was supposed to provide. Above the loop means the human sets the goals, defines the boundaries, and judges the outcomes the system produces — engaged oversight, not step-by-step approval. The shift only works when the architecture is trustworthy enough that you can step back without losing control. Without that architecture, you’re stuck in the loop forever, which is why so many agent pilots quietly become expensive supervised demos.

Getting there requires deciding, explicitly, how much autonomy each agent gets. A useful academic framework (Feng, McDonald, and Zhang, 2025) defines five levels by the role the human plays: operator, collaborator, consultant, approver, and observer — escalating autonomy as the human steps further back. You don’t grant an agent “observer-level” autonomy on a task that touches money or customers until it has earned it at “approver” level first. That progression — earn autonomy through demonstrated reliability, exactly as you would with a person — is the practical heart of AI governance.

And governance cannot be bolted on at the model layer. Anthropic’s own framework for trustworthy agents states it plainly: “the model layer alone cannot secure agentic AI.” Trust is a property of the whole system — the model, the tools it can reach, the data it can touch, the environment it runs in. We unpacked that five-principle framework — human control, value alignment, security, transparency, privacy — in what trustworthy AI agents actually require.

This is also the honest place to name where outside help earns its fee, because it’s the part that’s hardest to do from inside. Defining autonomy levels is straightforward on a whiteboard; enforcing them across real systems, building the audit trail a regulator or a board will accept, and holding the accountability cadence as agents multiply is organizational work that competes with everyone’s day job. It’s why some mid-market companies bring in a fractional Chief AI Officer — senior accountability for what the agents are allowed to do, without a full-time hire. The test to apply to us or any firm: does the engagement leave you owning the governance model, or dependent on the vendor to run it? If it’s the latter, keep looking.

How a Mid-Market Leader Should Actually Start

If you take one thing from this guide, take the sequence — because the order is where most companies go wrong. They start by buying agents. They should start by building the conditions agents need.

The first move is not a purchase; it’s a map. Pick one real process that costs you meaningful hours and where a mistake is cheap to catch. Understand it well enough to redesign it — because dropping an agent onto a broken process just automates the breakage faster. This is also where the honest “workflow or agent?” decision gets made: many of the highest-value wins are fixed workflows, not autonomous agents, and they’re cheaper and safer.

The second move is context and structure. An agent is only as good as what it knows about your business and how clearly its job is defined. The companies in MIT’s successful 5% had built the memory and context layers that let their systems improve; the 95% hadn’t. This is the substrate — a shared context layer and structured, maintained instructions for repeatable work — that turns a generic model into something that operates like it actually works at your company. It’s the difference between an AI that starts every task as a stranger and one that compounds. That substrate, plus governance and feedback loops, is what we mean when we talk about building the operating system a company runs on rather than a pile of disconnected bots — and it’s the core of the AI automation and agent systems we build with clients.

The third move is governance before autonomy, in exactly the progression above: start every agent as an approver-level assistant, watch it, and grant more autonomy only as it earns your trust. Cap what it can spend and touch. Keep a human above the loop.

Do those three in that order and you are building what the 6% of high performers built. Reverse them — buy agents, then scramble for context and governance after the invoice or the incident arrives — and you are building what 40% of companies will cancel by 2027. The technology is the same in both stories. The sequence is the whole difference.

That’s the work we do — for our own firm first, which is where we learned it, and then for mid-market companies who’d rather own the system that runs their business on AI than rent a collection of agents they can’t govern. Agentic AI is genuinely one of the most important shifts in a generation. It will also cancel most of the projects chasing it. Which of those two stories your company lands in has very little to do with the model you choose, and almost everything to do with what you build underneath it.

Sources

• Building Effective Agents — Anthropic (Schluntz & Zhang), December 2024 (workflows-vs-agents definition; start simple, escalate to agency only when needed)
• Trustworthy Agents in Practice — Anthropic, April 2026 (five principles; “the model layer alone cannot secure agentic AI”)
• Measuring AI Agent Autonomy in Practice — Anthropic, February 2026 (auto-approve ~20% under 50 sessions → 40%+ by 750; experienced users intervene more)
• Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 — Gartner, June 2025 (cancellation prediction; “agent washing”; ~130 of thousands of vendors genuinely agentic)
• Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026 — Gartner, August 2025 (40% by end 2026, up from under 5% in 2025)
• The State of AI in 2025: Agents, Innovation, and Transformation — McKinsey, November 2025 (23% scaling an agentic system; 88% use AI in ≥1 function; ~one-third scaling; 6% high performers)
• The GenAI Divide: State of AI in Business 2025 — MIT Project NANDA, July 2025 (95% no measurable P&L return; 5% of integrated pilots extract value; the organizational “learning gap”)
• State of AI in the Enterprise 2026 — Deloitte, 2026 (~three-quarters plan agentic within two years; only 21% report a mature governance model)
• Agentic AI Goes Mainstream but 94% Raise Concern About Sprawl — OutSystems, April 2026 (94% concerned about sprawl; 96% use agents; ~12% have a centralized management platform)
• Levels of Autonomy for AI Agents — Feng, McDonald & Zhang, arXiv, 2025 (five-level taxonomy: operator, collaborator, consultant, approver, observer)
• What Is Agentic AI? / Agentic AI vs. Generative AI — IBM (perceive → reason → act → learn loop; autonomy vs. content generation)
• Model Context Protocol — open standard open-sourced by Anthropic, November 2024 (standardized model-to-tool/data connection)
• Klarna AI assistant — OpenAI, 2024 (2.3M chats in month one; ~two-thirds of volume automated; 11 min → under 2 min) · How Klarna’s AI Agent Strategy Backfired But Became a Useful Lesson — Forbes, July 2026 (the walk-back; rehiring specialist human agents)

Frequently Asked Questions