Introduction

Veralith is an evaluation layer for retrieval-augmented (RAG) and memory-based AI systems. You send it each answer your system produced — the query, the context it retrieved, and the response it generated — and Veralith grades whether the answer was actually grounded in that context, classifies how it failed, and turns recurring failures into fixes your coding agent can open as pull requests.

Start here:

• Quickstart — install the SDK, add one line, and watch your first trace get diagnosed (about five minutes).
• Core concepts — trace → evaluation → failure cell → heal: the mental model the whole product runs on.
• Integration — log(), the @trace decorator, the LangChain adapter, and recipes for messy codebases.

What Veralith checks

For every answer, Veralith runs three judges and combines them into a single verdict:

• Sufficiency — did the retrieved context actually contain what was needed to answer?
• Faithfulness — did the response stick to that context, or did it fabricate (hallucinate) claims that aren't supported?
• Completeness — did the response cover the question, leave parts out, or pad it with unrequested extras?

Those three combine into one of six failure cells — a precise label for how an answer went wrong (for example, complete_ungrounded = a confident, fully-formed answer that's fabricated). See Core concepts.

The loop

1. Instrument — add one line to your RAG pipeline with the SDK. Each answer becomes a trace.
2. Evaluate — Veralith grades the trace asynchronously and assigns a failure cell. Nothing is added to your request latency.
3. Diagnose — recurring failures of the same root cause cluster into a heal card with a concrete, reviewable fix.
4. Heal — connect a coding agent (Claude Code, Cursor, Codex) over MCP and let it open the fix as a pull request.

Low-risk by design. The SDK is fail-safe (it can never break your app), fire-and-forget (zero added latency), and a no-op without an API key (safe to ship to every environment). See the SDK contract.