BM25 vs embeddings: hybrid wins for LLM trading retrieval
· 15 min read
Dense retrieval alone fails when queries contain rare, exact identifiers — tickers, CIK numbers, accession numbers, dates — where surface form is the meaning rather than a pointer to it. Adding lexical signal fixes that, and on this system's own corpus it helped on every query shape tested, not only on identifiers.