Why Your Enterprise AI Sucks Without a Proper Data Stack

AI is the hottest topic in every boardroom right now. Everyone wants in. But here’s the dirty secret a lot of companies are discovering the hard way: the biggest bottleneck isn’t the models, it’s their data.

Consumer AI tools are slick and fast. They impress users with instant answers and flashy demos. But try deploying that same AI at scale inside a real enterprise, and suddenly you’re knee-deep in legacy systems, siloed applications, and data formats that don’t talk to each other. It’s a mess.

Bavesh Patel, SVP at Databricks, puts it bluntly: “the quality of that AI and how effective that AI is, is really dependent on information in your organization.” Yet in most companies, that information is fragmented across dozens of disconnected systems. No unified view. No clear governance. No way for AI to pull trustworthy context.

Patel doesn’t sugarcoat the result: “terrible AI.”

That’s not a technical failure. It’s an infrastructure failure. And it’s becoming the defining challenge of this phase of digital transformation.

The real competitive differentiator for most organizations, Patel argues, is their own data plus the third-party data they can layer on top. But that only works if the data is consolidated into open formats, governed properly, and made accessible across functions. Without that foundation, you’re building AI on quicksand.

Rajan Padmanabhan, unit technology officer at Infosys, emphasizes that value focus is critical. Leading companies aren’t treating AI as isolated innovation experiments. They’re tying it directly to business metrics and using governance frameworks to kill what doesn’t work fast. That’s the difference between a proof-of-concept that goes nowhere and something that actually drives outcomes.

I’ve seen this pattern before. Companies rush to adopt AI because it’s trendy, then realize their data isn’t ready. They spend months cleaning, consolidating, and fighting internal politics over who owns what. Meanwhile, the AI projects stall or produce garbage outputs. It’s painful to watch.

What’s interesting is that both Patel and Padmanabhan see a huge opportunity in AI literacy for business users. “They’re very eager to understand how they should be thinking about AI,” says Patel. “What does AI mean when you peel the covers?” That’s the right question. Too many organizations skip the foundation and jump straight to shiny use cases.

The longer-term shift is even more significant. Padmanabhan describes moving from “a system of execution or a system of engagement to a system of action.” That means AI agents evolving from copilots into autonomous operators that manage workflows and transactions. But that only works if the data underneath is unified, real-time, and governed with precision.

The organizations that win will be the ones building that foundation now. Not next quarter. Not after the next board presentation. Now.

This isn’t glamorous work. It’s not going to make a flashy demo. But it’s the difference between AI that delivers real business value and AI that just wastes everyone’s time.