Core concepts
Everything in Veralith is built on one pipeline: trace → evaluation → failure cell → heal. This page defines each term.
Trace
A trace is one answer your system produced, captured as three parts:
| Field | What it is |
|---|---|
query | the user's question |
context | the chunks/memories your system retrieved to answer it |
response | the answer your system generated |
You create traces with the SDK. A trace is the unit everything else is computed from.
The three judges
Veralith grades each trace with three independent judges:
- Sufficiency — Did the retrieved context actually contain what was needed to answer the query? Low sufficiency means the retrieval step didn't surface the answer.
- Faithfulness — Did the response stay grounded in that context, or did it assert claims the context doesn't support? Unfaithful = hallucination (fabrication).
- Completeness — Did the response answer the whole question? It can be complete, incomplete (missed part), or have extra (unrequested/unsupported additions).
Note: faithfulness is grounding to the provided context, not truth in the world. If your context is outdated and the model repeats it faithfully, Veralith reads that as grounded — it flags fabrication and gaps, not stale sources.
The six failure cells
Completeness (complete / incomplete / extra) × faithfulness (grounded / ungrounded) gives six failure cells — a precise label for how an answer failed. This is the vocabulary you'll see all over the dashboard.
| Failure cell | Plain-language name | What happened |
|---|---|---|
complete_grounded | Healthy | Fully answered and grounded in context. The good one. |
complete_ungrounded | Hallucinated | Confident, complete answer — but fabricated beyond the context. |
incomplete_grounded | Retrieval gap | Grounded, but the context was missing part of the answer. |
incomplete_ungrounded | Missing + made-up | Worst case: incomplete and fabricated to fill the gap. |
extra_grounded | Over-answered | Grounded, but padded with unrequested extras. |
extra_ungrounded | Fabricated extras | Added claims not supported by the context. |
Tip: searching for "hallucination"? That's
complete_ungroundedandextra_ungrounded. Searching "retrieval gap"? That'sincomplete_grounded. The dashboard shows both the plain name and the cell id.
Healthy rate
Your healthy rate is the share of evaluated traces that landed in complete_grounded. The denominator is evaluated traces — an answer that hasn't finished grading (or whose eval failed) isn't counted as unhealthy, just not-yet-graded. So a backlog of pending evaluations won't drag the number down.
Honest abstention
Sometimes the correct behavior is to decline — the context genuinely doesn't contain the answer and the model says "I don't have that." Veralith detects this (retrieval covered nothing and the model fabricated no answer) and treats it as acceptable, not a failure. The Overview shows an abstention-adjusted healthy rate so a system that correctly abstains doesn't look broken.
Heal card
When failures of the same root cause recur, Veralith clusters them into a heal card — a single card describing the failure pattern, the affected traces, and a concrete, reviewable fix. Cards move through a lifecycle:
open → in_progress → pr_raised → resolved
(plus terminal states wont_fix, manually_fixed, superseded).
Self-heal over MCP
A heal card isn't just advice. Connect a coding agent — Claude Code, Cursor, or Codex — to Veralith over MCP (Model Context Protocol), and it can read the open heal cards, apply the fix in your repo, and open a pull request. See Integration to connect an agent.