Milvus

Each feature = what it is + why it's genuinely strong + what happens at the edge.

1. The full index spectrum: from FLAT to DiskANN Why it's real: the official index-selection guide covers FLAT (exact), IVF_FLAT / IVF_PQ / IVF_SQ8, the HNSW family (HNSW / HNSW_SQ / HNSW_PQ / HNSW_PRQ), SCANN, DiskANN, plus SPARSE_INVERTED_INDEX for sparse vectors; CPU / GPU / disk (NVMe) paths; AUTOINDEX lowers the entry barrier so newcomers don't have to read the index papers on day one. At the edge: indexes are built per segment, so fragmented small segments amplify total index overhead; HNSW memory runs ~1.5–2× the raw vectors (vendor / community figure); DiskANN saves memory but shifts cost to tail latency, disk IOPS, and build time — there is no free index, only different bills.
2. Storage-compute separation + segment / object-storage architecture Why it's real: writes go to the message queue (WAL) first, growing segments live in memory, sealed segments land on object storage as immutable files; QueryNodes load on demand; hot-cold separation holds naturally, and mmap can map raw data and indexes to disk to relieve memory pressure. At the edge: see Deep Waters 2, 5, 6 — the three classic confusions ("deleted data doesn't free space", "just-written data isn't searchable", "queries didn't get faster after scaling out") all come from this pipeline; any lag in growing / sealed transitions, flush, compaction, or handoff becomes a latency or space problem.
3. Dense + sparse / BM25 + scalar hybrid search Why it's real: one collection can hold multiple vector fields (multi-vector) with dense vector indexes, sparse vector indexes (SPARSE_INVERTED_INDEX, IP / BM25 metrics), and scalar inverted / BITMAP indexes side by side; a single hybrid_search with RRF / weighted reranking fuses semantic and keyword recall into one request; the BM25 function auto-derives sparse vectors from text at write time. At the edge: BM25 / analyzers must be declared at collection creation and cannot be added later; where filtering executes (before / during / after ANN) determines the candidate space and correctness; with highly selective filters leaving only hundreds of candidates, ANN graph traversal loses to a FLAT brute-force scan — "hybrid" is tuning work, not a free lunch.
4. Four tunable read-consistency levels Why it's real: Strong / Bounded Staleness (default) / Session / Eventually, built on GuaranteeTs; RAG workloads trade latency with Bounded, write-then-read flows use Session or Strong — clean tiering. At the edge: Strong folds the entire ingest → MQ → segment → QueryNode visibility lag into query latency (community test: 2× p95 at 50k QPS); third-party integrations may silently ignore your consistency_level (confirmed in a 2026-08 semantic-router issue); Milvus Lite only offers Strong — "tunability" is a distributed-system feature.
5. 3.0's lake-native and open table format (explicitly 3.x) Why it's real: 3.0 (GA 2026-07-16) introduces lake-native data access and open table formats with a stronger retrieval engine, aiming to let vector data live directly in data-lake formats and cut the "export a dedicated copy" ETL. At the edge: 3.x is a new major line — the Rust SDK declares 3.0 the first major line for 3.x while 2.6.x stays maintained; the 2.6 → 3.0.1 upgrade has three hard constraints (message-queue type immutable, chart version immutable, no image-only rollback after writes); milvus-backup supports 3.0 only from 3.0.1. Rehearse upgrade + rollback in a test environment before touching production.

Each entry = mechanism + what happens at the edge + what it means for selection. Evidence types labeled.

Deep Waters 1: Vector index algorithms — no "best", only different bills

• Mechanism: indexes are built independently per segment; index files land on object storage with their segments and are loaded by QueryNodes. HNSW graph index: M (connections per node) and efConstruction (build-time search width) determine graph quality vs. memory / build time; query-time ef determines recall / latency. IVF: nlist (cluster centers), nprobe (centers scanned) — larger nprobe means higher recall and linearly higher latency. DiskANN: graph on SSD, only compressed navigation structures in memory. FLAT: exact brute-force search.
• At the edge:
  • HNSW memory is ~1.5–2× the raw vectors (vendor / community figure): at billion-scale 768-dim vectors, index memory alone is a TB-level bill — which is why DiskANN and IVF_PQ exist; mmap shifts some pressure to disk at the cost of latency.
  • Fragmented segments (many small ones) multiply index overhead since each segment carries its own index; compacting small segments is index-cost optimization as much as space optimization.
  • Pick nlist wrong for IVF and no nprobe tuning will save you — clustering-stage information loss is irreversible; index parameters are fixed at build time, changing them means rebuilding and paying build cost again.
  • DiskANN tail latency follows the SSD: swap NVMe for ordinary cloud disks and P99 more than doubles; build time is also multiples of in-memory indexes.
• What it means: POCs must grid-test recall@k / P99 on real data distributions with real topK; index selection is a three-way negotiation between memory bill, latency, and build cost — no silver bullet.
• Sources: official docs (index_selection.md, incl. the HNSW tuning example M=16 / efConstruction=200 / query-time ef); community cost-optimization guide (memory multiple labeled as vendor / community figure).

Deep Waters 2: Deletion and compaction — "deleted" doesn't mean "gone"

• Mechanism: Milvus is append-only and supports logical deletion only — deletes write deletion markers (delta logs), and queries filter marked entities by timestamp; physical removal relies on compaction (triggered by DataCoord, executed by DataNode), which writes live data into new segments and retires old ones.
• At the edge:
  • Official release notes, verbatim: abusing deletion tanks search performance and explodes storage usage — every query must first walk the deletion markers.
  • The classic "deleted data doesn't free space" confusion: compaction is a background task with trigger thresholds and its own cadence; for GDPR right-to-erasure compliance you must verify physical removal end to end, not just query results.
  • Update = delete + insert: deletion-marker pileup in high-update workloads is far faster than intuition suggests; the 2026-07 mutable-columns design draft proposes column-level updates (write path described as "a delete with a payload") but is still at design stage — don't treat it as shipped.
  • Compacting segments triggers QueryCoord target updates and segment handoff (load the new segment before unloading the old) — not a fully invisible background task.
• What it means: workloads with heavy deletes / updates need compaction-strategy and marker-pileup load testing; compliance deletion must be verified down to the object-storage layer.
• Sources: official v2.0.0 release notes (logical-deletion warning); compaction design docs; archived load-segment-pipeline doc (compaction / handoff flow); mutable-columns design draft (2026-07, design stage).

Deep Waters 3: Scalar filtering and hybrid search — where the filter runs decides what you find

• Mechanism: scalar fields can carry INVERTED / BITMAP indexes; hybrid_search supports multi-path ANN with RRF / weighted reranking; filter expressions (expr / filter) compose with vector search; the partition-key feature (64 partitions by default when enabled) supports tenant routing.
• At the edge:
  • Filter placement changes the candidate space: with highly selective filters (hundreds of candidates left), ANN graph traversal's pruning advantage evaporates and a FLAT brute-force scan is faster and more accurate — "stronger index is always better" fails here.
  • IVF + filtering: nprobe scans cluster centers, and filtered-out vectors don't count toward recall — with extreme filter ratios the effective nprobe is diluted and recall drops silently; the fix is larger nprobe or switching to HNSW / FLAT, paid for in latency / memory.
  • Multi-tenant isolation (tenant_id filters) is the classic high-cardinality, low-selectivity filter: recall must be validated on real tenant data distributions, never on uniform-distribution benchmarks.
  • BM25 / analyzers must be declared at table creation and can't be added later — schema design has to anticipate "will we ever need full-text search", or it's rebuild-collection + reimport.
• What it means: hybrid-search POCs must test with real filter expressions and real data skew together; pure-vector-recall benchmarks say nothing about RAG production.
• Sources: official docs (index_selection.md, scalar-index docs); community practice (2026-08 BM25 hybrid-search design record, verified on v2.5.0 + SDK 3.0.4).

Deep Waters 4: Recall tuning — recall@k is a negotiation table, not a dashboard

• Mechanism: approximate retrieval trades accuracy for speed; the knobs come in three layers: index-build parameters (HNSW M / efConstruction, IVF nlist), query parameters (HNSW ef, IVF nprobe), and the data plane (segment size, compaction level).
• At the edge:
  • Changing a build parameter once means rebuilding the index once: raising efConstruction from 200 to 400 more than doubles build time — at billion scale that's a multi-day window.
  • Query parameters are the only "free" knob: ef / nprobe tune at query time, enabling per-tier traffic (high recall for core traffic, low latency for the long tail) — provided the SDK / gateway actually forwards search_params.
  • The official docs' "recommended values" are starting points, not destinations: defaults like M=16 / efConstruction=200 that suit 768-dim text vectors may be a different story on 2048-dim image vectors — dimension, data distribution, and topK jointly determine the optimum; no cross-scenario silver bullet.
  • Recall must be discussed together with filtering (see Deep Waters 3): a system scoring 99% bare ANN recall can drop below 90% under production filters — acceptance criteria should read "end-to-end recall@k with filters", not "bare ANN recall".
• What it means: write "recall@k / nDCG + P95 / P99 + real query set" into acceptance criteria; budget rebuild windows for index-parameter tuning.
• Sources: official docs (index_selection.md, HNSW tuning example); community consensus.

Deep Waters 5: Billion-scale expansion — three links between "can scale" and "scaled and serving"

• Mechanism: Proxy / QueryNode / DataNode / coordination scale independently; the segment is the smallest scheduling unit, with QueryCoord owning target management and handoff; sealed data sinks to object storage.
• At the edge:
  • Adding QueryNodes doesn't make queries faster by itself: new nodes must first load segments and indexes; before handoff completes the new node is decoration; segment migration during scale-out eats network and memory, so scaling at peak can wobble before stabilizing.
  • On the write side: growing segments seal via flush, and lagging flush piles up memory; a 3.0 dev build once showed write buffers waiting on slow segment sync with insert QPS dropping to 0 (issue #48974, final status pending verification) — the sustained-write backpressure chain is production's #1 killer.
  • External dependencies are the invisible ceiling: if etcd (metadata), Pulsar / Kafka (messaging), or object storage bottlenecks, more vector nodes buy nothing — "distributed" means bottleneck analysis spans five components.
  • Standalone (single node + MinIO / RocksMQ) and Cluster are two different ops species: moving from Standalone to Cluster is a topology change, not a config tweak.
• What it means: billion-scale POCs must load-test "sustained writes + queries + scale-out" concurrently; monitoring must cover MQ backlog, flush lag, and handoff progress — not just QPS.
• Sources: official architecture docs; archived load-segment-pipeline doc; GitHub issue #48974 (3.0 dev build, status pending verification).

Deep Waters 6: Consistency and visibility — GuaranteeTs sits between "write succeeded" and "searchable"

• Mechanism: writes enter the MQ (WAL) first, growing segments are queryable in memory, sealed / flushed data lands on disk; a query's GuaranteeTs decides "how new to wait for": Strong waits until QueryNodes' service time catches the latest timestamp; Session uses the session's last write timestamp (read-your-writes); Bounded (default) allows bounded staleness; Eventually skips the wait. Milvus Lite only offers Strong.
• At the edge:
  • Under the default Bounded level, "just-written data isn't searchable" is a feature, not a bug; switching to Strong fixes it but folds the entire ingest → visibility lag into every query's latency — a community test at 50k QPS saw Strong double p95.
  • Third-party integrations may swallow your consistency_level (2026-08 semantic-router issue #2821: Strong specified but not honored, later fixed) — "tunable" requires the parameter to actually propagate.
  • Under strong consistency, any ingest-side lag (MQ backlog, unsealed segments) becomes query latency directly — visibility delay is a full-pipeline problem, not query-side only.
• What it means: RAG "write-then-search" requirements need explicit visibility-latency validation in POCs; bake the consistency level into the SDK wrapper instead of relying on every caller to remember.
• Sources: official docs (tune_consistency.md, consistency.md; SDK docs note the Milvus Lite limitation); community issue (semantic-router #2821, 2026-08); community production practice (Datadog integration case).

Depth on the good stuff too. Each = why it's true + edges and limits.

1. Full index matrix — "no need to switch databases as you grow"
   • Why it's true: one system covers FLAT exact search through memory-saving IVF_PQ, high-recall HNSW, over-memory DiskANN, and sparse / BM25 full text; AUTOINDEX means newcomers don't read the index papers on day one.
   • Edges and limits: a full matrix means a large tuning surface means a large pitfall surface; per-segment index building amplifies index cost on small segments; changing an index parameter means rebuilding — picking wrong costs a rebuild window. And vendor / community "cut costs 80%" figures are specific-tuning-scenario claims, not general promises.
   • Source: official docs (index_selection.md); community cost-optimization guide. Observed version: 2.6.x.
2. Horizontal scaling as native architecture
   • Why it's true: storage-compute separation + stateless Proxy + independently scalable roles, segment-level scheduling; billion-scale is a design target, not a slogan.
   • Edges and limits: scaling out ≠ immediately serving (handoff / index loading); any bottleneck in external dependencies (etcd / MQ / object storage) nullifies new nodes; running Cluster at small scale is overkill, and Standalone eventually needs a topology change.
   • Source: official architecture docs; community consensus. Observed version: 2.6.x.
3. Hybrid search in a single request
   • Why it's true: dense + sparse + scalar filtering + RRF rerank complete in one hybrid_search, eliminating "query twice and fuse in app code"; the BM25 function auto-derives sparse vectors at write time.
   • Edges and limits: BM25 is fixed at table creation and can't be added later; filter × ANN recall interactions must be measured (see Deep Waters 3); RRF's k parameter needs tuning too.
   • Source: official docs; community testing (2026-08, v2.5.0). Observed version: 2.5+.
4. SDKs and AI ecosystem work out of the box
   • Why it's true: PyMilvus MilvusClient as the unified entry point, full coverage of Java / Go / Node.js / Rust / C++, RESTful v2; mature RAG-ecosystem integrations (LangChain / LlamaIndex); Milvus Lite (pip install) runs a prototype locally.
   • Edges and limits: ecosystem integrations may swallow advanced parameters (consistency_level non-propagation confirmed); Milvus Lite is Strong-only and feature-limited — production still needs the distributed deployment; 2.x → 3.0 requires switching SDK major lines.
   • Source: SDK docs; community issue (semantic-router #2821). Observed version: 2.6.x.
5. Complete observability and tooling
   • Why it's true: Prometheus + Grafana, Attu management UI (topology / slow queries / task queue / backups), OpenTelemetry / Jaeger tracing — problems are locatable by the book.
   • Edges and limits: Attu is a management UI, not a diagnostics platform; deep-water problems (stuck handoff, lagging compaction) ultimately need log and source reading; many metrics ≠ knowing which to watch — MQ backlog / flush lag need dedicated alerts.
   • Source: official docs; community practice. Observed version: 2.6.x.

• One-liner: A distributed system built for billion-scale vector search — the index matrix and horizontal scaling are genuinely strong, but complexity is split across coordination, messaging, object storage, segments, and index building; below pgvector's limits, those capabilities are just extra ops tax.
• Best for:
  • RAG / search / recommendation workloads at ten-million to billion vector scale, where single-node memory / HNSW can't hold;
  • Scenarios needing dense + sparse / BM25 + scalar-filter hybrid retrieval with independently tunable filtering and recall;
  • Teams needing index memory, query, and write to scale independently, or with K8s / SRE capacity to run a multi-component distributed system;
  • Teams happy to use Zilliz Cloud and outsource operations.
• Not for:
  • Small-scale RAG in the low millions — pgvector / single-node options are faster, cheaper, and simpler;
  • Teams deeply invested in Postgres relying on transactions / SQL / JOINs (Milvus has no SQL, no multi-statement transactions);
  • Teams with no dedicated platform / SRE staff hoping to "install and forget";
  • Scenarios with hard compliance-audit / TDE requirements restricted to self-hosted open source (no open-source kernel evidence found this round).
• Migration cost:
  • pgvector → medium-high: it's not just moving embeddings — schema, filter expressions, index parameters, and recall@k baselines must be rebuilt; no SQL means rewriting the access layer; compare recall@k / nDCG / P95 / P99 on a real query set before migrating, with dual-write + rollback plans.
  • Elasticsearch (vector use cases) → medium: BM25 / full-text capabilities map over, but ES DSL / sharding habits must be relearned; dense-vector indexes need rebuilding and tuning.
  • Self-built FAISS / ScaNN services → medium-low: index-algorithm concepts transfer; the main shift is from "index processes I manage" to distributed segment / compaction / handoff thinking; ops complexity rises, but self-built scheduling goes away.

• Version info: GitHub milvus-io/milvus releases (v3.0.1; 3.0 GA 2026-07-16); milvus-sdk-rust v3.0.0 release notes (first major line for 3.x; 2.6.x stays maintained)
• Architecture & components: official architecture docs; kubeblocks-addons milvus README (Standalone / Cluster dependency matrix: etcd + MinIO/S3 + RocksMQ / Pulsar / Kafka)
• Indexes & tuning: official index_selection.md (HNSW M / efConstruction / query-time ef tuning example; sparse / scalar indexes); v2.6.x reference/index.md (FLAT / IVF family / HNSW family / SCANN / SPARSE_INVERTED_INDEX matrix; SPARSE_WAND deprecated since 2.5.4)
• Consistency: official tune_consistency.md, consistency.md (four levels, Bounded default, GuaranteeTs); SDK docs (Milvus Lite Strong-only); third-party technical record (druce/dbengines: no transaction primitive, batch insert can partially succeed)
• Deletion & compaction: official v2.0.0 release notes (logical-deletion warning); load-segment-pipeline archived doc (compaction / handoff flow); mutable-columns design draft (2026-07, design stage)
• Hybrid search: official scalar-index docs; community testing (2026-08 BM25 hybrid-search design record, v2.5.0 + SDK 3.0.4); vector-anchored-join design draft (2026-07)
• Security: official adminGuide/tls.md (client TLS / mTLS / inter-component TLS); SDK best practices (RBAC); Zilliz official blog (Zilliz Cloud Audit Logs GA, extra billing on the managed side); Attu README (Attu's own audit-log boundary)
• Backup & recovery: zilliztech/milvus-backup README (online backup/restore, --rebuild_index, same-or-higher-version restores, 3.0 support from 3.0.1)
• Upgrades: official upgrade docs (upgrade_milvus_standalone-helm / cluster-helm / operator; 2.6.20→3.0.1 validated baseline; MQ-type and chart-version immutability; no image-only rollback after writes); milvus-helm README (2.5.16 mixCoordinator → 2.6.x path); OpenRAG migration docs (2.6 root → 3.x data-volume ownership change; legacy data migration requirement)
• Observability: official docs (Prometheus / Grafana / Attu); community practice (Datadog integration case, Strong consistency doubling p95; OpenTelemetry / Jaeger tracing blog)
• Complaint sources: GitHub issue #48974 (3.0 dev-build write backpressure, status pending verification); community blog (WPS 2.2.16→2.5.16 upgrade troubleshooting); semantic-router issue #2821 (consistency_level not propagated, 2026-08)
• Billing models: Zilliz billing references (Serverless vCU + storage; Dedicated CU × duration + storage; backup / network / audit-log metering boundaries) — no absolute prices recorded in this profile
• Chinese resources: official Chinese docs site (some pages "no Chinese version yet"); CSDN / Tencent Cloud developer community tutorials; discuss.milvus.io forum
• Next review: track 3.x lake-native progress and the 2.6.x maintenance branch after 3.x GA; re-verify whether the open-source kernel gains official audit / TDE capabilities; re-verify the final status of issue #48974