When does AI investment pay off in enterprise?

At CDO Vision San Francisco, the latest stop on the CDO Vision AI World Series, the conversation moved past the familiar optimism of AI adoption and into far more consequential territory. The panel discussion, titled "Beyond the Business Case: When Does AI Investment Actually Pay Off?", drew on hard-won experience from leaders across banking, fintech, and real estate to confront a question that boardrooms across the country are struggling to answer: how do you know when AI is actually working?

Every senior leader in enterprise AI today faces the same pressure. AI budgets are expanding. Pilots are multiplying. And yet, as the panel made clear, the gap between a compelling demonstration and a line item on the P&L is precisely where most AI ambition quietly stalls. The conversation that followed was not about technology. It was about leadership, accountability, and the strategic discipline required to turn experimentation into enterprise value.

The session was moderated by Sharad Mathur, a Data and AI Executive and Advisor with deep experience guiding organizations through large-scale transformation. The panelists brought complementary perspectives from across the industry: Gurpreet Atwal, Chief Data Officer for the Consumer and Small Business Bank at Truist, representing the complexity of regulated financial institutions; Abhinav Mangal, Vice President and Head of Analytics at EarnIn, offering a view from the fast-moving fintech world; and Gopal Renganathan, Senior Director of Data and Analytics at Anywhere Real Estate Inc., speaking to the challenge of driving transformation across a large and distributed enterprise.

The sharpest moment of the discussion came early, when Gopal described a conversation with a colleague at a major technology company, someone who had been celebrated by their CEO for ranking in the top ten for AI token usage across the organization. When Gopal asked what that meant for the business, the colleague had little to say.

The story landed because it captured something that many in the room recognized. Organizations have become adept at measuring AI activity: tokens consumed, models deployed, pilots launched, lines of code generated. What they have struggled to measure is whether any of it actually moved the business.

"If you measure how many tokens you use or how much data you use, those are activity-based measures. It needs to be outcome-based measures to actually have an impact."

Sharad reinforced the point by sharing a similar evolution at Zoom, where the engineering team had initially measured developer productivity by lines of code generated, a metric that quickly proved unreliable. The team eventually shifted to tracking features delivered to production, a measure that could be understood up and down the organization and tied directly to the end customer experience.

Abhinav offered a useful reframe. Rather than treating ROI as a binary outcome, he argued that the early objective should be directional: are we moving toward better customer experience, lower costs, faster operations? The full financial case, he suggested, would rarely be visible on day one. What mattered was whether the trajectory was right. As he put it, the parallel to regulated financial services was instructive, a model risk management review is not a loss-making investment. It is the cost of doing business.

If the first theme challenged how organizations measure AI, the second challenged something more fundamental: the assumption that a successful pilot is evidence of enterprise readiness.

"Don't show me pilots," Sharad said. "They're easy to build."

The panel was in broad agreement. AI pilots are almost always conducted in curated environments, bespoke conditions designed to maximize the likelihood of a positive result. The real test comes when organizations attempt to move those results across technology stacks, business units, and operational processes that were never designed with the pilot in mind.

Gopal added a perspective that cut across all three. The organizations most likely to scale AI successfully, he argued, were not the ones running the most pilots, they were the ones building horizontal data and platform layers that could outlast any individual experiment. When a tool or model changes, the investment is not lost if the foundation beneath it remains intact.

Abhinav noted the speed dimension. Portfolio management frameworks have always existed for evaluating technology investments. What has changed is the pace at which decisions need to be made and revised. The organizations that build in decision gates, clear criteria for continuing, pivoting, or stopping, are far better positioned than those that let pilots run indefinitely on the strength of early enthusiasm.

The third theme emerged organically from the scaling discussion, and it may have been the most forward-looking of the evening.

Gopal shared an observation that drew a knowing reaction from the room: a patent he had filed three years ago on large language model architecture had recently been granted. It was already obsolete. "Anything that is six months old," Gurpreet added, "is increasingly legacy."

In that environment, the panel argued, the traditional instinct to select the best available platform and build deeply around it carries significant risk. The organizations that are navigating AI most effectively are not the ones that have made the smartest technology bets, they are the ones that have built the organizational and architectural flexibility to change direction quickly.

At Truist, Gurpreet described a deliberate strategy of separating the data layer from the compute layer and keeping AI tooling deliberately flexible. Storing data in open formats ensures that it remains valuable regardless of which models or platforms consume it. Using frameworks that abstract away the underlying model, accessing the best available capability without committing to a single provider, allows the organization to move as the market moves.

"The thinking is not about the model," Gurpreet said. "It's about the outcome. I could go two versions back on a model and get as good a result for a tenth of the cost."

The governance dimension was equally important. Gopal argued that AI governance needs to operate at three levels simultaneously, strategic, operational, and tactical, and that the failure to maintain all three is what leaves organizations either overcommitted to the wrong platforms or too fragmented to act decisively.

The overarching message from the CDO Vision San Francisco panel was clear. Enterprise AI is moving out of its experimental phase and into one that demands a fundamentally different kind of leadership. The question is no longer whether AI can produce impressive results in a controlled environment. The question is whether organizations can measure outcomes meaningfully, scale what works operationally, and remain agile enough to adapt as the technology continues to evolve faster than any planning cycle can anticipate.

The leaders who will create durable value from AI are not the ones running the most experiments. They are the ones who have built the organizational discipline to know when an experiment has earned the right to scale, and the architectural foundations to do so without starting over.

As Sharad put it in his closing observation:

That warning, from someone who has seen organizations navigate multiple waves of enterprise transformation, may be the most practical piece of guidance that any AI leader can carry out of the room.