Vanquish

Retrieval is a correctness problem, not a search problem

Every coding agent has the same failure mode: it answers from the wrong paragraph with total confidence. The fix is not a bigger context window — it is making sure the model is looking at the right paragraph, with a citation, and that something checks its work before the answer is written. Vanquish is that something.

Vanquish is a Rust-native, GPU-accelerated corrective retrieval-augmented generation (CRAG) runtime exposed as an MCP server. It is deliberately not an answer generator. It sits between the agent and the corpus, and its only job is to return a graded, line-cited context envelope — grounded when the evidence is sufficient, insufficient with best-effort leads when it isn’t. The calling agent writes the answer; Vanquish decides whether the agent had any business writing it.

It currently grounds three corpora I actually work against daily: the Intel 64/IA-32 Software Developer’s Manual (~15k structurally-chunked sections), the Hyper-V Top-Level Functional Specification, and the UEFI 2.10 specification.

The route

One core loop, vanquish_route, with a model-free fast path and a full corrective path:

query
  → lookup-shaped? → entity-lane / section-ref bypass (~80ms, no models)
  → otherwise:
      F2LLM-v2 embed (ONNX)
      → Qdrant dense + Tantivy BM25 (+ optional codebase graph, capped & gated)
      → RRF fusion
      → jina-reranker-v3 listwise rerank (one forward pass, all candidates)
      → relative-threshold triage
      → auto-merge: co-located chunks collapse into whole-section spans
      → grader (Qwen3.5-0.8B, logit-confidence): sufficient?
          yes → envelope { status: grounded }
          no  → per-chunk CRAG refinement, prune noise
              → widen: hint-augmented query + entity-anchored traversal
                along parsed cross-references, relaxed triage
              → loop (bounded; early exit when nothing new surfaces)
      exhausted → envelope { status: insufficient, best-effort leads }

Three design choices carry most of the weight:

Structure-aware ingestion. Documents are chunked by heading hierarchy, every chunk prefixed with its breadcrumb. Tables and code fences stay atomic — a half-table is worse than no table. Cross-references (“see Section 28.2.3”) are parsed at ingest into a traversable graph, which is what makes the widen step symbolic rather than another blind embedding call: when the grader rejects a pass, Vanquish walks the document’s own citation structure. That is the “neuro-symbolic” in the tagline — neural retrieval, symbolic correction.

A grader that reads logits, not vibes. The sufficiency judge is Qwen3.5-0.8B running under llama.cpp GGUF, and its verdict is the first-token probability of the positive class — a calibrated logit-confidence read, not parsed free text. Bounded loops (3 max) with early exit keep worst-case latency sane.

A tool family, not one endpoint. vanquish_outline (TOC from breadcrumbs — navigate before querying), vanquish_lookup (exact-match entity lane for registers, MSRs, opcodes), vanquish_expand (fetch exact lines from a citation), vanquish_overview (RAPTOR-lite cached section summaries), vanquish_status, and vanquish_feedback — a usefulness signal that is the data source for future tuning. Cited spans are also readable as MCP resources (vanquish://source#L1042-L1069), so the client never needs filesystem access.

The model stack is pinned by measurement

Role	Model	Backend
Embedder	F2LLM-v2-0.6B	ONNX
Reranker	jina-reranker-v3 (BF16)	listwise, llama.cpp GGUF + safetensors projector
Grader	Qwen3.5-0.8B (UD-Q5_K_XL)	llama.cpp GGUF, ChatML, logit-confidence

None of these are defaults — each won a measured bake-off against its alternatives, and a shadow_reranker_backend config can A/B a challenger against live traffic (logging rank disagreements and Kendall tau) without touching served results. Storage is Qdrant for dense vectors, Tantivy for BM25, and SQLite for sections, entities, cross-references, feedback, and shadow-rerank disagreements. The retrieval plane is load-tested at 13M+ vectors — the corpora above are small by design (precision beats bulk for spec work), but the pipeline does not break a sweat at eight figures.

Evals are the gate, not an afterthought

vanquish-eval ships a golden set of hand-authored SDM queries with expected sections and fragments — grep-verified against the real corpus, CPU-only, CI-run — plus deterministic route-trace replay so any change can be regression-checked against recorded history, and compare to diff two runs. The rule is simple: nothing model-shaped — reranker swap, grader swap, HyDE, multi-query, merge policy — ships as the default until the harness says so with a number.

For concurrency, the server also runs as an HTTP daemon: one warm process serving N agents behind mandatory bearer auth, with per-session cancellation and an admission semaphore — so a fleet of agents doesn’t each pay for its own ~3GB of loaded models.

What I actually use it for

The corpus set is the tell: Intel SDM, Hyper-V TLFS, UEFI. Vanquish exists because hypervisor work means living inside three specifications totalling tens of thousands of pages, and “the agent hallucinated a VMCS field encoding” is a multi-day debugging sentence. Grounded retrieval with line citations turns spec consultation from a liability into a fast path — and the same runtime grounds anything else I point the indexer at.