engineering note / Verifiable AI Systems
Retrieval crash test: when similar means wrong
Why top-k does not replace a workflow benchmark
High semantic similarity can return a document that is wrong in time, procedure, or legal subject.
In brief
- Negative examples belong in the dataset.
- Retrieval metrics must match the user decision.
- A citation must be checked after retrieval.
Top-k is not a product outcome
A similarity ranking tells us which vectors are close to a query. It does not tell us whether a document concerns the right party, applicable law, procedural stage, or user question.
Recall and precision therefore need to connect to the decision the system supports. One ranking may be acceptable for exploration and unacceptable for drafting a justification.
A dataset with negative cases
The test set should include documents that look relevant but are wrong for one specific reason:
- an outdated version;
- another jurisdiction;
- a similar person or organisation name;
- a different procedural stage;
- a quotation whose missing context changes its meaning.
The evaluation unit
Passage-level evaluation is insufficient when the user needs a whole document, metadata, and relationships to other sources. The retrieval unit and evaluation unit must reflect the final artifact.
Verification after retrieval
Retrieval should be followed by a citation check: exact text, scope, source, version, and relationship to the claim. A model should not be the only verifier of material it previously generated.
Reading the benchmark
A score without corpus, query set, ground truth, and tolerated failure is only a number. There is no universal vector-database ranking; there are better or worse specified tests of a particular task.
01 / METHOD
The question the test must answer
A similar result can be wrong in time, procedure, or subject; the evaluation unit is therefore not the vector alone but the user decision and artifact it is meant to support. The value of the material is that a reader can separate the user question from a convenient tool metric and identify the point at which a result is no longer sufficient for the next stage of work.
02 / METHOD
Material and counterexamples
A negative set should include plausibly relevant documents from another jurisdiction, an obsolete version, a similar entity name, a different procedural stage, and a quotation stripped of context. Control material is not meant to confirm the author’s thesis; it should expose a case in which a plausible result is dangerously misleading for the workflow at hand.
03 / METHOD
Control method
The protocol connects retrieval, reranking, metadata checks, and citation fidelity. Measures such as Recall@k are meaningful only where a real truth set and task unit are documented. The account includes step order, input, and result trace so an independent person can inspect what was measured and what the procedure does not measure at all.
04 / METHOD
Acceptance matrix
An acceptance matrix states which failure is tolerable in exploration and which one blocks use in the preparation of professional material. Criteria are stated before the result is read, so a single successful example cannot substitute for control of a critical failure type.
05 / METHOD
Handoff and rerun
After a corpus, embedding-model, or filter change, the test is rerun with the same identified cases and a record of discrepancies. Reproducibility also means showing why a subsequent run differs from an earlier one and who decides whether a new version may be used.
06 / METHOD
Boundary of the conclusion
Any synthetic example is marked as synthetic and does not pose as a public benchmark. The article does not claim a model result without a source-backed dataset. The material is therefore methodological: it does not pose as a universal recipe or as an independent confirmation that has not been performed.
07 / METHOD
A worked check
A control example can deliberately contain two similar passages: one from the correct source version and one with the wrong jurisdiction or date. If the result cannot distinguish them through citation and version, similarity is not yet evidence of correct retrieval. This example is synthetic, not a production measurement.
08 / METHOD
Source and review record
The review package records query identifier, corpus version, retrieval configuration, expected passage, returned passage, and human decision. Recall@k is meaningful only when such a labelled set exists; without it, a number should not be presented as a quality measure.
LIM
Limitations and scope
Benchmark results depend on domain, corpus, and evaluation unit; there is no universal vector-database ranking.
SRC
Sources and external edition
Original or earlier Medium material
AI tools assisted with structure and editing. The final text underwent human editorial review of facts, sources, conclusions, and attribution.
LOG
Change history
- First expanded edition in the owned archive.
- Review of structure, limitations, and evidence links.