Every model is only as intelligent as the data it learns from. That sounds obvious. It is also where most organizations get the hardest part of AI wrong.
Model architecture gets the attention. Data quality determines the outcome and data quality is itself a function of two things most teams underinvest in: how efficiently that data is labeled, and how reliably it is generated at the volume modern AI systems require. Get either wrong, and no amount of architectural sophistication recovers what was lost upstream.
Maitrik Patel, an engineering manager at Apple working on AI systems at consumer scale, has spent his career on the discipline that makes model architecture possible in the first place: producing data models can actually learn from, efficiently and at scale.
The annotation problem
Annotation is not a clerical task. It is thousands of small human decisions, what counts as correct, what counts as ambiguous, aggregated into a signal a model will treat as ground truth. When that process is inconsistent, the model does not fail randomly. It fails in ways that are hard to trace back to their source.
Patel’s work has long centered on this layer: building infrastructure that lets human judgment scale reliably, across languages and use cases, without losing the consistency that makes a label trustworthy in the first place.
What happens when models generate their own training data
As large language models matured, a second source of training data became viable: synthetic examples generated by the models themselves, rather than labeled entirely by hand. Patel’s more recent work has engaged directly with this shift, and with the question it raises about reliability.
That question is also the subject of his academic research. As a co-author of ASTRA-bench (arXiv:2603.01357), a benchmark evaluating tool-use AI agents under real-world personal context conditions, Patel has studied what happens when an agent’s training or evaluation data does not accurately reflect the messiness of real user context. The finding is that agents rarely pause to ask. When information is missing, they proceed anyway. When information conflicts, many struggle to reconcile it. Both failure modes look like confidence from the outside, but the paper traces them to different root causes.
Why production feedback closes the gap
Benchmarks and synthetic data can only anticipate so much. The conditions a model actually encounters after deployment, the edge cases, the ambiguous requests, the ways real users phrase things differently than training data assumed are a separate and often richer source of signal. Teams that treat post-deployment data only as something to monitor are leaving the most realistic data they will ever have on the table.
Patel’s recent work has engaged with this problem directly: building the infrastructure that both surfaces production issues and feeds that behavior back into training as structured input for the next iteration of the model.
The discipline is still forming
The discipline of treating data as seriously as model architecture is still young. A handful of teams have built the infrastructure and the habits to do it well. Most have not, and the gap between the two groups is becoming one of the clearest predictors of which AI systems actually hold up in production.
“Model performance is a lagging indicator,” Patel has said. “Data quality is the leading one. Most teams obsess over the lagging indicator and ignore the lead.”
