Give Your AI Agent a Business Semantic Layer: Ontologies, OBDA, and MCP in Practice

Data teams stuff table schemas into the prompt and let a large language model write SQL directly. It's the most common approach today—and the most dangerous. Because business meaning has never lived in the schema.

A wrong number that looks perfectly professional

Picture a data-analysis agent wired into three systems at once: a CRM, a billing system, and customer support. A business colleague asks:

How much net revenue did active customers in East China contribute last quarter?

The agent returns a number quickly, nicely formatted, with a clearly stated definition. But it may have used the wrong customer table, counted refunds as revenue, and guessed the definition of "active customer." Worse, nobody can explain how that number was calculated.

The trouble is that all three systems have a table called Customer, and they mean completely different things:

• A CRM customer is a company that has signed a contract.
• A billing-system customer is a payment account.
• A support-system buyer is a person who called in.

One word, three different business entities. The model sees table names and column names. It does not see that distinction.

Why putting the schema in the prompt isn't enough

The common practice today is to put the database schema into the prompt and let the model read table names, column names, and foreign keys directly. The model does "see" the schema. But it cannot see the knowledge that actually determines whether the answer is right:

• Does "active customer" mean someone who ordered in the last 90 days, or someone with a valid contract?
• Should net revenue deduct refunds?
• Which provinces does East China actually include?

This knowledge is scattered across metric docs, wikis, and the data team's heads—some of it never written down at all. The model can only guess. And when it guesses wrong, the SQL still runs and a result still comes back. That's what makes this class of error so dangerous: it looks correct.

So why not stuff every document into the prompt? No context window, however large, can hold the documentation for hundreds of systems. And placing several definitions of the same word side by side makes the model more confused, not less.

A gap of more than threefold

In 2023, Juan Sequeda and colleagues ran a benchmark on an enterprise database in the insurance industry. When GPT-4 translated questions directly into SQL, execution accuracy was only about 16%. When the data was first represented as a knowledge graph through an ontology and mappings, and GPT-4 generated SPARQL instead, accuracy rose to about 54%—more than triple.

Agents face a higher bar still: they don't just answer questions, they act on the answers. We want their decisions to be accurate, consistent, and explainable—summarized as the ACE principle.

The solution is to place a semantic layer between the data and the agent, writing business meaning in a machine-readable form. The agent consults the semantic layer first, then touches the data.

The term "semantic layer" is not new. BI tools have had one for years, mainly governing metric definitions and dimensions so that revenue is calculated the same way in every report. The agent's semantic layer goes further: it must let the model understand concepts, relationships, and rules—and reason over them.

A complete agent semantic layer usually contains three kinds of context: metrics, ontology, and documents. The division of labor is clear:

• Metrics: calculations like "net revenue" that must come out identical every single time.
• Ontology: descriptions of concepts, relationships, constraints, and the business meaning of fields.
• Documents: prose such as policies, processes, and exceptions—for example, a revenue recognition policy that explains the reasoning behind a rule and lets an answer cite its basis.

The three complement one another rather than replace one another. Underneath is a neuro-symbolic idea: the large language model understands natural language and proposes candidate answers, while the ontology and reasoner constrain, validate, and explain using deterministic logic. One is flexible; the other is reliable.

How an ontology lets an agent query the right data: OBDA and virtual knowledge graphs

The key technique is the virtual knowledge graph, also known as ontology-based data access (OBDA).

The idea: data stays in its original database—no migration. We write only a mapping that says which table and which columns each concept and relationship in the ontology corresponds to. At query time, the system automatically rewrites a query against the ontology into SQL against the database.

Back to the customer example. The ontology defines a class, "Contracted Customer," mapped to the CRM's Accounts table, and another class, "Billing Account," mapped to the billing system's Billing Accounts table. A relationship, "has account," connects the two and corresponds to the join key between the tables. This kind of mapping has a W3C standard: R2RML.

With it, what "customer" means in the CRM versus the billing system is no longer hidden inside some engineer's SQL. It becomes an explicit, reviewable declaration. An ontology holds more than classes and relationships: you can give each concept a description, synonyms, and business terms. Whether a user says "customer," "client," or "contracted party," it all maps to the same concept.

The full flow: the agent receives a natural-language question and generates SPARQL against the ontology. The open-source engine Ontop rewrites the SPARQL into SQL according to the mappings. The SQL runs on the original database. The model only ever faces one clean set of concepts—not 700 tables directly.

For example, querying contracted customers in East China takes only three lines in SPARQL: find all contracted customers, retrieve their region, and filter for East China. The rewritten SQL, by contrast, has to join three tables and translate provinces into region codes.

This brings two benefits. First, the model is more likely to get it right, because it sees business language like "contracted customer" and "has account" rather than a pile of abbreviated column names. Second, every step is traceable—which concepts the question used, which tables those concepts map to, and which SQL finally ran.

Layered resolution: precise first, flexible second

With metrics, ontology, and documents in place, which should the agent reach for when a question arrives? One practical design is layered resolution: start with the most precise method, and hand off to the next layer when it can't cope.

Layer one is metrics. If the question happens to be a defined metric—"What is total net revenue?"—the system directly executes that metric's pre-written SQL. The result is deterministic; everyone who asks gets the same number. But our question adds several qualifiers: last quarter, East China, active customers. The metric's SQL is fixed and can't take those conditions; it would return only an unfiltered total. So the question goes to layer two.

Layer two generates a structured query from the ontology. It does not discard the metric: the official definition of net revenue is passed to the model along with the question, and the model adds the time, region, and active-customer conditions on top of that definition, then generates SQL through the virtual knowledge graph. The definition comes from the metric; the concepts come from the ontology.

The ontology can also check things before execution. Suppose it states that the subject of the "has account" relationship must be a contracted customer. If the generated query tries to join an individual from the support system as a contracted customer, it violates that constraint and can be blocked before it runs.

Layer three is knowledge retrieval. If a user asks, "Why are refunds deducted from revenue?", the answer is in no table—it lives in a document like the revenue recognition policy. The system retrieves and traverses the knowledge graph and documents, and the large language model synthesizes an answer with citations. This is the GraphRAG we touched on last time.

The three layers fall back in order: the further down you go, the more flexible but less deterministic and slower it gets. Whichever layer is used, the generated SQL passes through a SQL firewall before execution: only query statements are allowed, and the tables and columns it can access are limited by the user's permissions.

Building the ontology with AI, with experts as gatekeepers

That all sounds fine—but where does the ontology come from? The traditional approach takes knowledge engineers and domain experts months of working together to organize concepts and write mappings, which is why many teams balk at the effort.

Large language models have changed this. You can now have AI read the schemas, comments, and sample data in the data catalog, draft an ontology and mappings, and hand them to experts for review.

AWS's open-source Context Ontology Accelerator is one such tool, and its flow has three steps:

• Scan: connect to data sources, discover schemas, enrich metadata, and optionally import unstructured documents.
• Model: derive an ontology, define metrics, and build a unified semantic graph.
• Serve: deliver context to agents through queries and graph traversal.

When deriving the ontology, a "generate-plus-review" method works well: process tables in foreign-key dependency order, first retrieve related concepts from the existing ontology, have one model generate an ontology fragment, have another model review and correct it, and finally merge it into the core ontology.

There is also an alignment step called grounding: mapping newly generated classes to existing ontologies such as FIBO and schema.org to prevent the same concept from appearing twice. This step can't be done by name alone—in insurance, Policy means an insurance policy, but in an ontology from another domain, Policy can mean a policy or regulation. Same name, different meaning; you need semantic judgment to tell them apart.

The generated ontology then goes through validation: a reasoner such as HermiT checks logical consistency, then quality metrics and common modeling errors are examined, and finally domain experts review, modify, and approve it item by item before it reaches production.

AI drafts; humans gatekeep. Descriptions and synonyms edited by experts replace the AI-generated versions, and old versions are kept as history. That way the model always sees exactly one definition, never two conflicting ones.

The accelerator is deployed inside your own AWS account: Amazon Neptune stores the semantic graph, OpenSearch handles vector retrieval, and Amazon Bedrock provides the models. The ontology is saved in W3C standard formats and can be used in other tools as well.

Handing the semantic layer to the agent: MCP and identity-aware governance

Once the ontology is built, how do you hand it to the agent? The most common way today is MCP, the Model Context Protocol. The semantic layer packages its capabilities as a set of tools that any MCP-capable agent can discover and call.

Take Context Ontology Accelerator as an example. It offers six tools: two for exploration—listing which metrics exist, and describing the classes, properties, and data tables in the ontology—and four for end-to-end natural-language queries—translating to SPARQL, retrieving document snippets, and traversing entity relationships on the semantic graph.

A useful pattern is "explore first, execute second": the agent calls the ontology-describing tool to understand the difference between contracted customers and billing accounts, then issues the query. It's like a new employee reading the business glossary before looking at the data.

Governance matters just as much. An agent acts on behalf of a specific user, so every call is authorized against that user's identity, not the agent's own. No authorization means deny by default—tables the user can't see, the agent can't see either. Every call leaves an audit record: who initiated it, which concepts were used, which SQL ran, and where the result came from. If something goes wrong you can reconstruct it, and the compliance team can verify exactly what the agent accessed.

This direction is becoming infrastructure. Cloud services are attaching business descriptions, data-usage notes, and business annotations directly to data catalogs and storage, publishing them in open formats so that agents can fetch governed context at runtime.

Five practical recommendations

First, start small. Don't try to build the whole company's ontology from day one. Pick one high-value domain—customers and revenue, say—and make it deep and accurate. Over-modeling is one of the most common beginner mistakes.

Second, prepare a set of demo questions. Pick a few dozen representative questions from real business and write down the gold answers. This serves as both acceptance criteria and a regression test: rerun it after every change to the ontology, mappings, or metrics, and check whether accuracy has dropped.

Third, materialize only when you truly need to. When cross-domain integration genuinely requires it, materialize that part. If the mappings get convoluted, build some views at the source that align with the ontology structure.

Fourth, do governance from day one. The ontology needs versioning and an approval flow, with draft and production kept separate and rollback always available. Large language models can draft ontologies and mappings, but every change needs human review.

Fifth, value observability and explainability. Record which layer each query took, which concepts it used, and how long it took. Attach citations and a reasoning chain to answers. Users trust an agent's conclusion only when they can see the basis for it. At the same time, follow open standards such as RDF, OWL, SPARQL, R2RML, and RML as much as possible, so you don't get locked into one tool.

Closing

Agents get the answer wrong because business meaning isn't in the schema. A semantic layer fills that gap with metrics, an ontology, and documents. A virtual knowledge graph maps the ontology directly onto existing data, with no migration. Layered resolution balances determinism and flexibility. AI drafts and experts gatekeep, dramatically shortening the time to build an ontology. MCP and identity-aware governance hand all of it safely to the agent.

With an ontology, an agent's answer isn't only computed correctly—it can also explain its basis.