So many AI pilots follow the same arc. Week one: everyone’s excited, the demo works, the model answers questions correctly. Week four: the project is “paused” while someone tries to figure out why the AI told the VP of Sales that churn was 4% last month and told the CFO it was 11%, using the same underlying tables.
Nobody wants to say the quiet part out loud, so it gets filed under “AI readiness.” But the semantic layer is the actual problem, and it was the problem before anyone bolted a chatbot on top of it. The AI just made it impossible to ignore.
The Pattern
The story is almost always the same shape. A company wants to give business users a natural-language interface to their data, or hand an agent the ability to answer “how are we doing this quarter” without someone writing SQL. The model itself is rarely the blocker; newer models are more than capable of translating a decent question into a decent query.
What kills the timeline is discovering that “active customer” has four definitions across finance, sales, and product, and nobody agreed on one before the AI showed up and started confidently picking one for them. I watched a team burn weeks not on model tuning, but arguing about which of eleven columns named some variant of “revenue” was the one an agent should actually query. That’s not an AI problem, that’s a data-modeling problem that’s been sitting there for years, quietly tolerated because humans writing dashboards knew, by word of mouth, which column to avoid.
“AI Readiness” Is a Marketing Term for “Your Data Model Is a Mess”
…and thus vendors sell “AI readiness assessments.” It sounds like a new problem that needs a new solution, which is a much easier sale than “your metrics layer needs to be rebuilt, and that’s going to take a quarter.” But strip the AI branding off and you’re looking at the same data-modeling discipline teams have needed forever: consistent business definitions, a single governed place those definitions live, and a way to keep them from drifting apart as tables and teams multiply.
We’ve done this dance before. The ’90s and 2000s had “enterprise data warehousing” as the fix. The 2010s had “master data management” and later the dbt-driven “metrics layer” movement. Each wave rediscovered that the hard part was never the pipes, it was agreeing on what a “customer,” a “sale,” or an “active user” actually means, then encoding that agreement somewhere a machine can read it. AI didn’t invent this problem. It just removed the human buffer that used to quietly smooth over it.
Why LLMs Make This Impossible to Hide
When a human analyst writes a query, they bring tribal knowledge with them. They know Finance’s “revenue” table has a known bug, or that the “active_users” flag hasn’t been updated since a migration in 2023. An LLM doesn’t know any of that unless you tell it, and it will answer with total confidence either way. That’s the part that scares executives once they see it: a wrong number delivered with zero hesitation is more dangerous than a wrong number a junior analyst delivers with a shrug and a “don’t quote me on that.”
I wrote about Snowflake Cortex AI a while back, and the tools in that space (Snowflake semantic views, dbt’s semantic layer, AtScale, etc) all exist to solve exactly this: give the model a governed, curated set of metrics and business terms to query against instead of turning it loose on raw tables. That layer is the actual product. The chat interface on top of it is the easy 20%.
What a Semantic Layer Has to Do
Strip away the tooling and a working semantic layer needs to answer three questions the same way, every time, for every consumer:
- What does this metric mean? Not “revenue” in the abstract, but the exact filter logic, the exact grain, the exact exclusions (refunds in or out? which currency conversion?).
- Where does it live, and is it current? One governed table or view per concept, not eleven candidates with slightly different names.
- Who’s allowed to see it, and at what level of detail? Row-level security and masking policies don’t go away because an agent is asking the question instead of a person. If anything those policies matter more, because the agent will happily hand a summarized number to someone who shouldn’t have it unless the policy stops it first.
None of that is an AI capability, it’s data governance and data modeling. The only thing that changed is the audience for that governance now includes a model that will act on bad definitions instantly and at scale, instead of a human who might pause and ask a Slack channel first.
What Actually Works
The engagements that go well all do the unglamorous thing first: they treat the semantic layer as the deliverable, and the AI interface as a thin client on top of it. In practice that means:
- Identify the metrics that actually get asked about (usually a shorter list than people expect)
- Get the business owners of those metrics (not just the data team) to sign off on one definition each, in writing, somewhere versioned.
- Build those definitions into a governed semantic layer rather than letting each AI tool reinvent its own.
- Point the agent at that layer, not at raw tables, and treat “can’t answer without a governed definition” as a correct answer, not a failure.
This is also, not coincidentally, the same groundwork that makes agentic workflows tractable. If you’re evaluating agent frameworks (see AI Agentic Frameworks Comparison) every one of them still has to hand the agent the tools to query real data at some point. An agent with a clean semantic layer underneath it is dramatically easier to build, test, and trust than one wired straight to a warehouse full of ambiguous tables.
Bottom Line
If your AI initiative is stalling, resist the urge to blame the model, swap vendors, or buy a bigger context window. Go look at whether “active customer,” “revenue,” and “churn” have real and clear definitions (side note: “churn” in particular can mean very different things in different industries).
The unglamorous fix is still the fix: build the semantic layer, govern it like it matters, and let the AI be the interface instead of the excuse. Companies that do this in the right order ship AI use cases in weeks. Companies that skip it spend quarters debugging why their very capable model keeps giving very confident wrong answers.
Leave a Reply