Retrieval at Scale | Drop for 2026-01-27

TL;DR

Since Jan 15, 2026: Weaviate 1.35.4 shipped performance and networking optimizations (notably avoiding cross‑node vector transfers and faster compressed‑HNSW rescoring); Milvus 2.6.9 added highlight scores and operational fixes; Elastic shared a practical knob (“best_compression”) that can speed search, not just save space; a new study clarifies how to use cased LLM backbones for learned‑sparse retrieval (lowercase at ingest/query); and a TREC ToT report shows hybrid fusion + learned/LLM reranking wins on “vague recollection” queries.

Weaviate 1.35.4: less cross‑node vector traffic, faster compressed‑HNSW rescoring

  • Key facts and current state of the topic
    • Weaviate continues to harden high‑QPS retrieval: the 1.35 line already brought Flat‑index RQ at GA, server‑side batching (preview), and ACORN‑by‑default for filtered ANN. The .4 patch adds concrete serving optimizations. (weaviate.io)
  • Important context and background information
    • Cross‑node payloads and verification passes dominate tail latency and cost in multi‑vector and hybrid setups; trimming network I/O and rescoring work directly improves p95/p99.
  • Recent developments or changes
    • v1.35.4 (Jan 26) avoids sending vectors with cross‑node search unless needed, optimizes rescoring for compressed HNSW, and improves backup robustness (splitting large shards). Expect lower network overhead and steadier tails under load. (github.com)

Milvus 2.6.9: highlight scores, segment reopen, and stability fixes

  • Key facts and current state of the topic
    • Milvus is widely used as a first‑stage ANN/hybrid engine; recent 2.6.x releases added JSON shredding, NGRAM, boost ranker, geospatial, and hybrid performance knobs. (milvus.io)
  • Important context and background information
    • Better observability of matched spans and safer segment lifecycle handling reduce variance in hybrid pipelines (vector + lexical/filter rerank).
  • Recent developments or changes
    • v2.6.9 (Jan 16) adds highlight scores for search results, supports reopening segments after data/schema changes, improves storage version handling, and updates dependencies; upgrade notes also call out a MinIO bump for security. (milvus.io)

Elastic: “best_compression” as a performance lever, not just storage

  • Key facts and current state of the topic
    • Elastic’s Labs post (Jan 23) shows that enabling best_compression (zstd in modern stacks) can measurably improve search performance, not only reduce disk footprint—useful for Lucene/Elasticsearch‑based vector/lexical hybrids. (elastic.co)
  • Important context and background information
    • Coupled with recent Lucene 10.x gains and ACORN‑style filtered ANN, compression that cuts I/O can increase effective probe budgets under the same latency. (lucene.apache.org)
  • Recent developments or changes
    • Consider testing best_compression alongside BBQ/RaBitQ quantization settings and filtered‑ANN parameters to balance recall/latency/cost in production clusters. (elastic.co)

LSR best practice: using cased LLM backbones (lowercase your text)

  • Key facts and current state of the topic
    • A Jan 24 study finds learned‑sparse retrieval built on cased LLM backbones performs worse by default than uncased—but simple lowercasing at ingest/query eliminates the gap. (arxiv.org)
  • Important context and background information
    • Many newer high‑quality backbones are cased‑only; this result preserves compatibility with LSR (e.g., SPLADE‑style or LLM‑based sparse) without custom vocab tricks.
  • Recent developments or changes
    • Actionable guidance: standardize lowercasing in your LSR preprocessing/serving path when adopting cased backbones to keep effectiveness and latency predictable. (arxiv.org)

Fusion retrieval + learned/LLM reranking for “tip‑of‑the‑tongue” queries

  • Key facts and current state of the topic
    • The TREC ToT 2025 report (posted Jan 21) shows a two‑stage system—hybrid candidate generation (LLM‑based, BM25, dense) plus learned and LLM rerankers—achieves strong recall/NDCG on vague‑recollection queries. (arxiv.org)
  • Important context and background information
    • For ads/search use cases with multi‑intent or underspecified queries, fusion in the gather stage and learned reranking can recover recall without exploding ANN budgets.
  • Recent developments or changes
    • Consider topic‑aware multi‑indexing and lightweight learned rerankers before invoking heavy LLM reranking to control cost while retaining ToT robustness. (arxiv.org)