The Metric
Domain-Conditional Retrieval Precision (DCRP@k) measures retrieval quality within a semantic domain, not across domains. Standard Recall@10 asks "did you find legal documents for a legal query?" DCRP@10 asks "did you find the right legal documents?"
Corpus
Two production vector collections:
- legal_docs_v2 — 244,000 vectors (768d, nomic-embed-text). Emails, filings, financial statements.
- case_docs — 1,700,000 vectors (768d, BGE + SPLADE++). Full legal document corpus with hybrid search.
Status: Experiments running.
Results will be populated here once
run_dcrp.py completes against both collections. The experiment tests PCA reduction at 16, 32, 64, 128, 256, 384, 512, and 768 dimensions.
Expected Results
Based on the theoretical framework (PCA variance ≠ semantic information):
- Inter-domain Recall@10 at 16d: ~95% — consistent with prior claims. The system can still tell law from medicine.
- DCRP@10 at 16d: ~40-55% — catastrophic for real-world use. The system can't distinguish specific legal concepts within the legal domain.
- Crossover point: ~128-256 dimensions — where DCRP degradation exceeds 10%.
- Variance explained at 16d: 85-95% — confirming prior findings, but demonstrating this metric is uncorrelated with DCRP.