Semantic Layers & Ontologies: Cheaper, Sharper, More Truthful LLMs

Ask an LLM "What was last quarter's revenue?" and it has to guess what revenue means in your company: gross or net, which currency, which tables. A semantic layer removes the guessing.

Definitions

• An ontology is a formal model of the concepts in a domain and how they relate: Customer places Order, Order contains Product, Invoice belongs to Customer.
• A semantic layer maps those business concepts to real data sources, with governed definitions such as "Net Revenue = gross sales minus refunds and discounts", plus units, synonyms and access rules.

Together they act as a single, shared vocabulary between your data and your AI.

Why this makes outputs more correct

Without structure, the model must infer meaning from raw column names like amt_2 and from whatever text it was trained on. With a semantic layer:

• Terms are unambiguous. "Revenue" resolves to exactly one governed metric.
• Queries are constrained. The model picks from valid metrics and dimensions instead of inventing SQL against unknown tables, so a wrong join or non-existent column becomes far less likely.
• Answers are consistent. Two people asking the same question get the same number.
• Results are checkable. Each answer can cite the metric definition and source.

Why it reduces hallucination

Hallucinations happen when the model fills gaps with plausible-sounding text. An ontology shrinks the gaps: facts are retrieved from a knowledge graph as explicit relationships, and the model is instructed to answer only from them. If the graph has no answer, the system can say so instead of fabricating one. Validating output against the ontology (is this entity real? is this relation allowed?) catches many remaining errors before a user sees them.

Why it reduces token cost

Dumping entire schemas or document collections into every prompt is expensive. With a semantic layer:

1. The system identifies the concepts in the question.
2. It retrieves only the few relevant definitions and facts, a small, precise context.
3. The model reasons over that compact context and returns a structured answer.

Shorter prompts mean fewer input tokens, lower latency and less cost per query, and less irrelevant text for the model to be distracted by. Exact savings depend on your data and workload, so we measure them on your own queries.

A practical architecture

1. Model the domain: entities, relationships, metrics, synonyms.
2. Connect them to databases, APIs and documents.
3. Expose the layer to the LLM as tools (look up a metric, traverse a relation, run a governed query).
4. Validate and log every answer with its sources.
5. Evaluate on a fixed question set and iterate.

This is the foundation of the AI systems we build at Culpine: language models that speak your business's language, and can show their working. Curious how it would apply to your data? Get in touch.