Why the agentic AI era demands a different kind of data architecture – and what to do about it.
At Google Cloud Next last week, Andi Gutmans, VP and GM of Data Cloud at Google, said something that got less attention than it deserved: “The data architecture has to change now. We’re moving from human scale to agent scale.”
That’s a short sentence with long consequences. Because what it implies for the data infrastructure most organisations have spent the last five years building is genuinely uncomfortable.
The modern data stack – dbt transformations, a Snowflake or Databricks warehouse, orchestrated pipelines, BI dashboards – was designed around a clear assumption: the primary consumer of your data is a person. Someone composing a query, reviewing a dashboard, taking a decision. The whole architecture optimised around that. Scheduled jobs, SQL-first thinking, curated datasets handed off to analysts. For what it was built to do, it works well.
Then you introduce AI agents. Systems that run continuously, reason across multi-step workflows, and query data autonomously at any hour without a human composing the request. The same stack that served your analysts starts to struggle – not dramatically, at first. The bottlenecks emerge gradually. And often at exactly the moment you’re trying to show your board what AI is actually doing for the business.
Nearly two-thirds of enterprises have experimented with AI agents. Fewer than 10% have scaled them to deliver tangible value. Eight in ten cite data limitations as the core obstacle. That’s not a model problem. It’s a data architecture problem.
Those are McKinsey’s figures, published this month. They land differently when you’ve been in rooms where engineering leaders are convinced their AI investment is underperforming because of the model choice. It almost never is.
The gap nobody planned for
The pipeline-centric data stack was built for predictability. You knew roughly when data would be consumed, by whom, and in what format. Governance, access controls, semantic definitions – these were important but manageable. A small team of data stewards could curate the catalog, label tables, define business terms, and keep the glossary roughly up to date.
Agents don’t operate like that. They need to retrieve, reason about, and act on data at a pace and volume that manual curation cannot keep up with. Worse, they need context. Not just raw data – the meaning behind it. What does “active customer” mean in your CRM versus your billing system? Which revenue figure is the authoritative one when three tables all have a column called “revenue”? An analyst picks that up through onboarding, conversation, and tribal knowledge accumulated over months. An agent has none of that unless it’s been explicitly encoded somewhere.
Google’s response to this at Cloud Next was the Knowledge Catalog – an automated semantic layer that infers business logic from query patterns rather than relying on humans to define it. The point isn’t that you need Google’s specific product. The point is that without something equivalent, your agents are working without the context they need. And agents working without context produce expensive mistakes, often with confidence.
What actually needs to change
This is not a case for pulling out everything you’ve built. The lakehouse architecture, the transformation pipelines, the BI tooling – most of it remains useful. The problem is the layers that are missing around it.
Semantic context has become infrastructure
Historically treated as a documentation exercise, semantic metadata is now a functional requirement. When an agent queries your data, it needs to understand what it’s looking at without a data steward to ask. Business glossaries, entity relationships, and access-aware retrieval logic need to be machine-readable – not buried in a Confluence page that was last updated eighteen months ago. This is a shift in how your data team spends its time, not just a tooling change.
Data quality is a reliability problem, not a reporting one
Many organisations have quality checks that surface issues on a dashboard. A person reviews that dashboard and decides whether to act. Agents can’t pause their workflow to check a dashboard. They’ll consume whatever is in the pipeline and proceed. If your quality enforcement sits downstream of your consumption layer, that’s a serious structural problem. The checks need to move upstream, and remediation needs to be automated – or at minimum, clearly flagged before data reaches the agent layer.
Cross-cloud data gravity is a hidden cost at agent scale
Most enterprises have data spread across AWS, Azure, and GCP. For human-driven analytics, the egress fees from querying across environments are manageable. Agents querying continuously – potentially thousands of times a day – turn those fees into a budget line item. Open table formats like Apache Iceberg, which allow cross-cloud querying without relocating data, are worth evaluating now, before the invoice arrives.
The pipeline-writing model is on borrowed time
Data engineers who spend the majority of their time hand-crafting transformation pipelines are watching their tooling automate significant portions of that work. The shift isn’t primarily about headcount – it’s about where engineering effort should be directed. The teams pulling ahead are moving engineers toward outcome definition and architectural governance. Those that aren’t are accumulating technical debt that will need to be paid off at exactly the wrong moment.
The broader picture
None of this means your data team has done anything wrong. The modern data stack, as it existed through most of the 2020s, was a genuine step forward from what came before. The challenge is that the target has moved.
The infrastructure race used to be about storage cost and query speed. Then it became about reliability and governance. Now it’s about context – making data legible not just to people, but to the systems that will increasingly act on your behalf.
The organisations that will struggle are those treating their data stack and their AI programmes as separate conversations. They aren’t. Your data architecture is now a precondition for your AI performance. Whether your agents produce trusted outputs or hallucinated ones, whether they scale or stall at proof-of-concept – all of it traces back to the quality, structure, and accessibility of your underlying data.
According to McKinsey, the organisations that have successfully scaled agentic AI are those that treated data architecture modernisation as inseparable from their AI programmes, not as a preceding IT project that someone else owns.
The vendors are moving fast. Snowflake has Cortex, Databricks has Unity Catalog, Microsoft has its Fabric semantic layer, and Google just announced Agentic Data Cloud. These tools are useful. But they surface on top of whatever foundation you’ve built. If your underlying data quality is poor or your semantic definitions are thin, no vendor product fixes that automatically. The tooling accelerates good architectural decisions. It doesn’t substitute for them.
Q&A: Building for Agent Scale
Does this mean we have to rebuild our entire data stack?
No. The lakehouse architecture, your transformation pipelines, and your BI tooling are still useful. What changes is the layer above and around them: semantic context, automated quality enforcement, and governance structures that work at machine speed rather than human speed. Most organisations can modernise incrementally rather than starting from scratch.
We’ve only just implemented dbt and Snowflake. Are we already behind?
Not necessarily. A well-structured modern data stack is a better starting point than fragmented, siloed systems. The question is whether you’ve built the semantic and governance layers that agents require. If your data catalog is thin and your quality checks are largely manual, that’s where the work is. The pipes are probably fine. The context around them needs attention.
What’s the practical difference between a traditional data catalog and what agents actually need?
A traditional catalog tells humans what data exists and roughly what it means. An agent-ready semantic layer goes further: it encodes business definitions, entity relationships, access policies, and ranked retrieval logic in a format that can be consumed programmatically. Agents don’t read documentation. They need that knowledge embedded in the architecture itself.
Should we wait for our platform vendor to solve this?
Snowflake, Databricks, and Google Cloud are all actively working on it. But these tools perform on top of whatever foundation you’ve already built. If your underlying data quality is poor, or your semantic definitions are incomplete, no vendor announcement changes that. The tooling accelerates good decisions – it doesn’t replace them. Waiting is a decision, and it has a cost.
How do we know if our current stack is ready?
The honest test: run an agent against it. Give an AI system access to your data with a real business question and see what happens. Where it produces confident but wrong answers, or fails to find data that clearly exists, you’ve found your gaps. Most organisations discover these faster than they expected. A structured readiness audit – rather than an ad hoc experiment – is more systematic, but the experiment at least tells you where to start.
Working Through This With Vertex Agility
The shift described in this article – from a human-optimised data stack to one that can support agent-scale consumption – is a conversation we’re having with technology leaders across a range of industries right now. The specifics vary: some are dealing with poor data quality that their AI programmes have exposed, others have solid foundations but missing semantic layers, and some are trying to figure out how to modernise without stopping ongoing delivery.
Our Data Consultancy practice works with organisations to build scalable data architectures across the full range: from pipeline design and lakehouse implementation through to the semantic and governance layers that AI systems need to perform reliably. If your data stack is established but your AI programmes aren’t producing the results you expected, the gap is usually architectural rather than a problem with the models themselves.
Our AI Consultancy sits alongside this. We work with teams on AI strategy and implementation, custom model development, data engineering for AI, and the governance frameworks required for responsible adoption at enterprise scale. Having both disciplines in one practice means the data work and the AI work stay connected – which is the core argument this article has been making.
If you want an independent view of where your current stack stands, we offer a free AI Readiness Mini Audit on our website. For something more substantive, get in touch with us directly below.