Generative AI pilots often succeed in controlled environments, but fail when they meet real enterprise data: legacy systems, inconsistent definitions, missing lineage, weak governance, and teams unprepared to operate the platform at scale.
In this interview, Vishnu Bapathi, Lead AI Engineer at Boston Consulting Group, discusses why effective AI transformation depends on production-grade data engineering – from cloud-native platform architecture and data quality guardrails to automated governance, data contracts, and BCG’s Digital Data Platform approach.
According to a recent MIT study, up to 95% of generative AI projects do not lead to real business transformation. From your perspective as a lead implementing AI in large enterprises, does this figure reflect reality? Where exactly do these projects tend to break down?
That number doesn’t surprise me. If anything, it’s generous – because it counts projects that delivered some value as successes. In my experience across Fortune 500 clients in financial services, healthcare, energy, and telecom, the failure usually isn’t dramatic. It’s quiet. A pilot works. A demo impresses leadership. Then the project enters the real enterprise environment and slowly dies – not because the model was wrong, but because the infrastructure underneath it couldn’t support production-grade operation.
When you say “the infrastructure underneath,” what do you mean in practical terms?
The breakdowns happen in three predictable places. First, the data foundation. Teams build a strong model on clean sample data, then discover that production data is fragmented across legacy systems, inconsistently defined, or stale by the time it reaches the model. I’ve seen optimization models making daily operational decisions on sensor data that hadn’t been updated in 72 hours. The model worked perfectly. The data feeding it was broken. Second, governance. In regulated industries – banking, healthcare, energy – you can’t deploy AI without audit trails, lineage, access controls, and explainability. These have to be designed into the architecture from the start.
Third, organizational readiness. The client team expected to own the system may not have the skills, documentation, or processes to operate it. The AI works when the consulting team runs it, then degrades after handover. That’s a delivery model failure.
Many companies successfully launch pilots but fail to scale their solutions. What is the key technical reason behind this gap – models, data, or architecture?
It’s data and architecture. Almost always. A pilot uses a clean, bounded dataset. It runs in a notebook or sandbox environment. The model performs beautifully. Then you scale it, and suddenly you’re ingesting data from 33 branches running different legacy systems. “Revenue” means different things in different tables. Schemas change, pipelines break, and data arrives late or inconsistently. The model hasn’t changed. But the data environment is completely different from the pilot – and nobody designed the architecture to handle that complexity. On a route optimization platform I led, we processed 170 million rows daily. The model itself was sound, but scaling it required solving problems the pilot never surfaced: memory management after Spark-heavy phases, cluster sizing, and guardrails that catch flatline sensor readings before the model treats them as valid inputs.
These are data engineering and architecture problems. The model was the easy part.
You have spent many years building data platforms in regulated industries. Based on your experience, where do bottlenecks most often arise in AI transformations – in the models themselves or in data engineering and architecture?
Data engineering and architecture, by a significant margin. The model is typically 10–15% of the effort in a production AI system. The other 85% is getting the right data to the right place, at the right time, in the right shape, with the right governance. In regulated industries, the bottlenecks compound. At a financial services client, model development couldn’t begin until we resolved data lineage, because regulators needed to trace inputs to outputs. At a healthcare client, governance requirements meant months of work on access controls and de-identification layers before any model was trained. At an energy client, data was scattered across mainframes, IoT sensors, and cloud platforms with no unified semantic model.
So the common mistake is underestimating the engineering foundation?
Exactly. Organizations invest heavily in data science talent and model development, but underinvest in the data engineering and platform architecture that makes those models useful. They hire PhDs to build models, but not enough data engineers to build the plumbing those models depend on. I recently published a framework called “Agentic Data Contracts” that addresses this directly – using AI agents to negotiate and enforce data quality agreements between data producers and consumers. The core argument is that data quality governance needs to be automated and architectural. If the data foundation is unreliable, no amount of model sophistication will save the project.
Could you share a typical example from your practice where an AI initiative failed to deliver results? What exactly went wrong at the level of data or architecture? Of course, without disclosing any confidential information.
I’ll share a composite that represents a pattern I’ve seen more than once. A logistics client had an optimization model that planned daily delivery routes. The model was technically excellent – strong solver, well-tuned parameters, good backtesting. But in production, trucks were showing up at locations where customers didn’t actually need deliveries. The operations team was losing trust in the system. When we investigated, the model wasn’t wrong. The data was. Around 20% of the operational failures are traced to sensor data quality issues. Sensors were reporting tank levels that hadn’t changed in days – flatline readings that the model interpreted as stable inventory, when the sensor had stopped updating. Post-delivery readings were also inaccurate, causing the model to miscalculate future consumption.
The architecture had no guardrails: no freshness check, no anomaly detection for flatline patterns, no fallback logic for stale readings. The model consumed whatever data the pipeline provided. We built data quality guardrails – delivery-based tank resets, 48-hour fallback logic, flatline suppression, and post-delivery validation. These were projected to eliminate 50–60% of data-driven incidents. We also reduced runtime by 67% and cut daily compute costs by 40%. The lesson: the absence of a data quality layer between raw data and the model was the real failure.
Now, looking at the opposite side: what key engineering principles distinguish projects where AI actually begins to create real business value?
Three principles consistently separate the projects that deliver value from those that stall.
First, start from the business decision, not the data. Successful projects begin with: “This person needs to make this decision by this time – what data and intelligence do they need?” On a commodity trading platform I architected, we started with the trader’s daily workflow: what decisions they make at 7 AM, what data informs those decisions, and where the gaps are. That led to a GenAI-enabled platform combining ML forecasting with LLM-driven intelligence because the use case demanded it. Second, design the data architecture for production from day one. The semantic layer, freshness metadata, lineage tracking, and access controls must be part of the initial architecture. On a healthcare data platform, we designed nine business domains with 43 entities across bronze, silver, and gold layers before any model was built. That made downstream use cases trustworthy. Third, plan for human adoption. The best platform is worthless if the client team cannot operate it after the builders leave. Successful projects treat knowledge transfer, runbooks, documentation, and team upskilling as core deliverables.
Have you experienced a situation where you expected an AI project to succeed, but it did not? What did you take away from that experience as an engineer?
Yes. Early in a healthcare engagement, we designed what I thought was an architecturally sound data platform – Snowflake-centered, multi-layer, well-governed. The technical design was strong. But we underestimated organizational complexity. The client operated across 33+ branches, each with legacy AS400 and mainframe systems. The data wasn’t just fragmented technically – it was fragmented culturally. Different branches had different definitions for the same business concepts. Teams with deep legacy knowledge were protective of that knowledge and resistant to standardization. We faced legacy knowledge hoarding, poor documentation, skill gaps, siloed branch operations, ownership confusion, hardcoded business logic, manual workflows, risk aversion, and governance gaps. The lesson was humbling: architectural correctness is necessary, but not sufficient. You can design the perfect three-layer data model, but if the organization is not ready to adopt it, the platform will not deliver value.
You operate at the intersection of deep engineering and business needs. What is more challenging for you – building the right architecture or convincing the business to use it correctly?
Convincing the business to use it correctly. Without question. I can design a data architecture in weeks. I can evaluate cloud platforms, choose tools, design ingestion and transformation layers, and build governance into the system. That’s hard technical work, but it’s bounded. Getting a business to adopt what you’ve built is unbounded. It requires understanding organizational dynamics, navigating competing priorities, building trust with stakeholders, and translating technical decisions into language that a CFO or COO can act on. On a pricing platform transformation, the technical work – migrating engines, building 36 Power BI dashboard views, designing semantic models – was complex but manageable. The harder work was shifting the organization from “BCG runs the models” to “our team owns and operates the models.” That required hands-on knowledge transfer, not just documentation.
The engineers who succeed in enterprise AI understand that architecture is the starting line, not the finish line.
You have worked on advancing a platform-based approach to data, including DDP at BCG. How has the practice of AI implementation changed when companies move from fragmented solutions to platform-based architectures?
The change is fundamental. When organizations operate with fragmented solutions – separate systems for each use case, business unit, or geography – every AI initiative starts from zero. You spend the first months just getting data into a usable state. The cost compounds: fragmented solutions create fragmented data, fragmented insights, and fragmented decisions. A platform-based approach – what we call DDP, or Digital Data Platform, at BCG – inverts this. You build the foundation once: unified ingestion, governed transformation layers, standardized semantic models, centralized access, and security. Then every AI use case starts from a stronger position. The data is already integrated, governed, and documented. I’ve seen this with energy and utilities clients. Before the platform approach, each AI initiative – gas supply planning, inventory forecasting, cost-to-serve optimization – had its own pipeline, definitions, and quality problems. After moving to a cloud-native platform architecture, with standardized ingestion and governance built into the layers, use cases that previously took months could be delivered in weeks.
The most important shift is cultural. Business teams stop asking “can we get the data?” and start asking “what insight can we derive?” That is when AI implementation becomes a business capability.
Looking ahead 3–5 years: what needs to change in how companies approach AI – at the engineering level, not just the strategy level – so that transformations finally begin to deliver sustainable results?
Three shifts need to happen. First, data engineering has to become a first-class investment. Many companies still put most of their AI budget into model development and much less into the data foundation. That ratio should change. Models are becoming commoditized; the real advantage is proprietary data, governed platforms, and operational data quality.
So the model itself is no longer the main differentiator?
Exactly. The differentiator is whether the company can feed that model with reliable, governed, timely data in production.
Second, governance needs to become automated and architectural. Committees, policies, and periodic audits cannot scale when AI systems operate at machine speed. We need data contracts in CI/CD pipelines, automated quality checks before data reaches a model, and lineage tracked by the system rather than in spreadsheets.
Third, companies need to build for adoption from day one. Too often, they spend 18 months on platform development and two weeks on “change management.” Successful teams design training, workflows, incentives, and ownership transitions alongside the technical architecture. Most companies are still building AI on fragmented, ungoverned data and wondering why results are inconsistent. The tools already exist. It’s a prioritization gap – and closing it is an engineering leadership problem.
