RAG

navra includes a built-in RAG pipeline through the navra-rag crate. It combines FTS5 full-text search with sqlite-vec vector similarity, fuses results with Reciprocal Rank Fusion (RRF), reranks with an optional ONNX cross-encoder, and gates low-confidence results. All storage is SQLite -- no external vector database required.

Pipeline overview

Document → Chunk → Embed → Index (SQLite + sqlite-vec)
                                      ↓
Query → Embed → FTS5 + Vector → RRF Fusion → Rerank → Gate → Results

Each stage is independently configurable:

Indexing documents

Chunking

The chunking engine splits documents into overlapping chunks:

1. Split at paragraph boundaries (double newline)
2. Merge short paragraphs up to target_size
3. Split long paragraphs at sentence or code boundaries
4. Add overlap between adjacent chunks

For code files, the engine detects function/struct/impl boundaries and splits there instead of at sentence endings.

Configure chunking with ChunkConfig:

Breadcrumb injection

For Markdown documents with headings, inject_breadcrumbs() prepends the heading hierarchy to each chunk's content. A chunk under # Project > ## Setup gets the breadcrumb "Project > Setup" prepended, so the embedding captures structural position alongside content.

Section pointers

inject_section_pointers() annotates each chunk with its parent section's byte range. On retrieval, the caller can expand a chunk hit to the full section for more context.

Graphability filtering

predict_chunk_value() scores chunks from 0.0 to 1.0 based on structural signals. Chunks under headings like "Appendix", "License", or "Changelog" score low. Code-only chunks without prose score at most 0.5. Set graphability_threshold to skip low-value chunks during indexing.

Document type detection

detect_document_type() classifies input as Code, Prose, Markdown, or Structured (JSON/YAML/XML) based on line pattern heuristics. This can be used to select chunking strategies per document.

Using the rag_index tool

Tool: rag_index
Parameters:
  path: "/home/user/docs/architecture.md"

The tool reads the file, chunks it, generates embeddings via the configured model, and stores everything in the chunk store.

Querying

Hybrid search

search_hybrid() runs both FTS5 and vector search, then combines results with RRF. Documents appearing in both channels get boosted scores. This outperforms either channel alone -- FTS catches exact keyword matches while vectors catch semantic similarity.

HyDE (Hypothetical Document Embeddings)

search_hybrid_with_hyde() adds a third search channel: the embedding of a hypothetical ideal answer generated by an LLM. HyDE embeddings are closer to stored answers than question embeddings, improving recall for question-style queries. The caller generates the hypothetical document and embeds it.

Cascading confidence gates

search_hybrid_cascading() runs FTS5 first. If the top BM25 score is strong enough, it skips vector search entirely. If vector search runs but the top distance is already very close, it skips cross-encoder reranking. This reduces latency without sacrificing quality when early stages are confident.

Configure cascade thresholds:

Using the rag_query tool

Tool: rag_query
Parameters:
  query: "How does authentication work?"
  limit: 5

Returns ranked chunks with source path, chunk index, and distance score.

Cross-encoder reranking

After the initial vector search retrieves approximate nearest neighbors, an optional cross-encoder scores each (query, candidate) pair for fine-grained relevance. navra uses ONNX models (e.g., cross-encoder/ms-marco-MiniLM-L-6-v2) loaded at startup.

The reranker:

• Tokenizes all query-document pairs
• Runs batched ONNX inference (one call for all candidates)
• Falls back to sequential scoring if batching fails
• Replaces vector distances with cross-encoder scores

When a reranker is active, the pipeline over-fetches 4x the requested limit from the vector index to give the cross-encoder enough candidates.

Graceful degradation

load_reranker() tries to load the model files. If they are missing or fail to load, it returns a NoopReranker that passes candidates through unchanged. The pipeline always works -- reranking is an optimization, not a requirement.

Confidence gating

GatedReranker wraps any reranker with a confidence threshold. After reranking, it computes the mean absolute score of all results. If the mean falls below the threshold, it returns an empty result set (abstention). The caller checks for empty results and surfaces a configurable abstain message.

Default threshold: 0.4.

Semantic query cache

QueryCache detects paraphrased queries by comparing embedding vectors with cosine similarity. When a new query is semantically close to a cached query, the cached results are returned without re-running the search pipeline.

Configure caching:

Enable caching on a ChunkStore:

ChunkStore::open("index.db", 384)?
    .with_query_cache(QueryCacheConfig::default())

Agentic retrieval

The AgenticRetriever performs multi-step retrieval with query decomposition and self-correction.

Query decomposition

decompose_query() splits compound queries on conjunctions ("and", "then", "also") and routes each sub-query to the appropriate search strategy:

Negation support

Natural language negation is detected and translated to FTS5 NOT operators. Patterns like "not", "without", "except", "excluding" are extracted:

Input:  "auth not OAuth"
FTS5:   "auth NOT OAuth"

Temporal predicates

Dates in ISO 8601 format after keywords like "after", "since", "updated after" are parsed and routed to filtered search with SearchFilter.min_updated_at.

Self-correction loop

If initial results fall below a relevance threshold, the retriever extracts follow-up terms (function names, type names, file paths) from the initial results and refines the query. It retries up to max_hops times, merging all results with RRF.

MCP tools

The RAG module exposes four MCP tools:

All tools respect navra's permission system. rag_index requires read permission on the file path. rag_query requires the search operation.

Data erasure

The chunk store supports right-to-erasure operations:

• delete_by_source(source_id) -- remove all chunks for a document
• delete_by_content_match(query) -- remove chunks containing specific text (e.g., a person's name)

Both methods delete the associated embedding vectors alongside the chunk rows.

Quality metrics

evaluate_quality() checks structural integrity and size compliance of chunked output:

• Block integrity: detects split code fences, mid-list breaks, and mid-table breaks
• Size compliance: fraction of chunks within target size +/- 50%

Use these metrics to tune ChunkConfig for your document corpus.