What It Will Take for Agentic AI to Earn Enterprise Trust

The question enterprises are asking about AI has changed. For two years it was "what can generative AI produce?" Now it is "what can AI do on its own?" That shift, from generating content to executing work autonomously, framed a panel of data and AI leaders from healthcare, pharmaceuticals and banking. Their shared verdict: more autonomy demands more governance, more trust and more human judgment, not less.

The moderator, Nirav Parikh, Head of Enterprise Data & AI at Mass General Brigham, set the tone early. His team's current priority is not more agents but a secure, well-governed foundation for running them in production. The rest of the conversation explained why.

The consultant and the high performer

The most memorable definition came from Subhasis Bhattacharya, Executive Director of Engineering at athenahealth, whose remit includes data engineering and site reliability engineering. Generative AI, he said, is a consultant who tells you what should be done. Agentic AI is the high performer who actually does it.

His team is already living that difference. Agents now read log files, identify issues and trigger self-healing fixes, sharply cutting mean time to recover. Data pipeline failures that once pulled engineers into war rooms, such as an upstream schema change breaking a CI/CD job, are increasingly diagnosed and handled by agents.

Sid Chopra, Vice President and software engineering leader at Citizens Bank, who joined the panel as a late stand-in, drew the same line in different terms.

Alexander Shvets, Head of Patient Analytics, Data Science & AI at Ipsen, pushed the definition further. Agentic AI is rarely one agent. It is a cohort of agents, some executing, some checking quality, each focused on part of a task. The hard problems are giving them enough context and orchestrating them so errors and hallucinations do not compound.

Bhattacharya added a caveat that rarely makes the keynote slides. Agentic systems consume far more tokens than a single prompt and response. They suit high-value, high-volume work; for low-utilisation tasks, the economics may not hold.

Parikh's summary was crisp. Generative AI produces outputs such as text, code or summaries. An agentic system manages goals, plans actions, incorporates feedback and coordinates decisions with minimal human involvement.

What gets automated, and what stays human in 2030

The more interesting answers came on what will not be replaced. Bhattacharya offered a historical arc. The industrial revolution devalued human muscle. AI is now devaluing raw cognitive speed, including coding. What gains value is what remains: empathy, care and being human.

Chopra made the same point about trust. Banking is a relationship business, and every customer wants to be understood. AI can power the assistants and the workflows, but humans must stay on the loop, shaping how it all fits into the customer's journey.

Shvets brought the regulated-industry reality. Healthcare data is usually collected for another purpose, and one patient's record is like Swiss cheese, full of holes. Interpreting it takes approximations and judgment. He noted that regulators are moving towards "human in the lead", not merely in the loop. In his view, agents will support decisions in these settings, not make them autonomously.

Parikh closed the round with a line that framed the rest of the discussion: AI cannot take accountability. Medical diagnosis, ethical decisions and crisis response will need a person who answers for the outcome, in 2030 and well beyond. The panel also expected work itself to fragment, with more people selling skills to several organisations rather than holding one job.

Human in the loop: a permanent fixture or training wheels?

The sharpest question was whether human oversight is permanent or only lasts until agents earn trust. Self-driving cars once seemed impossible, Parikh noted, yet many people now ride in them without incident.

Shvets was optimistic that some tasks will be fully automated. But he invoked the TV doctor Gregory House, whose maxim was that everybody lies. Medical data depends on what patients report, and that will never be fully reliable. His bigger worry was deskilling. If people trust autopilot completely and it fails, they may no longer know how to drive. Keeping humans sharp is part of the job.

Bhattacharya offered the most practical framework: decide by risk and confidence. If the risk is low and the model's confidence is high, hand the task to the agent. If the decision touches someone's livelihood, has regulatory exposure or could hurt the business, keep a human in the loop. He used insurance as the analogy: if an autonomous car crashes, the insurer still holds the person behind the wheel responsible.

Chopra reframed autonomy as something granted, not owned. Treat an agent like an intern or a child: give it a defined level of autonomy for a task, and take it back when the task ends. You do not hand a ten-year-old the car keys, and even licensed drivers get penalised for mistakes. In this view, autonomy is always provisional.

The winners will be the best governed, not the best equipped

Each panelist left the room with one idea.

Bhattacharya compared the moment to the early cloud era, when every initiative had to be "cloud first". Today it is "AI first". AI will feel mature when people stop talking about it, as they did with cloud. Until then, be clear-eyed about where it works well and where humans are still better.

Shvets urged leaders to start with the business. AI is a tool for answering key business questions, and every deployment should be judged by the outcome it supports.

Chopra encouraged people to be "extremely selfish" in using AI: with so many digital tools and assistants available, use them for every stakeholder and every interaction.

Parikh delivered the line that best summed up the session. The value of agentic AI's autonomy far exceeds content generation. But the winners will not be those with the best model or the most agents. They will be the organisations with the strongest governance, the cleanest data foundations, and clear lines of trust and accountability, whatever technology they choose.

That is a useful corrective to an industry fixated on capability. Agents that can act are now within reach of most enterprises. Whether they should act, and who answers when they get it wrong, remains a human decision. The organisations that design for that question first are likely to move fastest from pilot to production.