Meet LEANN: An Open-Source Vector Index that Shrinks a 60M-Chunk RAG Index from 201 GB to 6 GB

What changed

UC Berkeley rolled out LEANN, an open-source vector index designed to drastically lower storage needs for large retrieval-augmented generation (RAG) systems. LEANN compresses a 60 million-chunk index from 201 gigabytes down to just 6 gigabytes. It does this by pruning the graph structure typically used in indexes and deferring embedding recomputation to query time. This means it uses much less disk space by not storing every embedding outright.

Why builders should care

Most large vector search indexes rely on storing massive amounts of precomputed embeddings, which racks up enormous storage and maintenance costs. LEANN shifts the storage-versus-latency trade-off by leaning hard on on-demand recomputation of embeddings at query time. It matches nearly the same recall accuracy as state-of-the-art HNSW indexes at 90% recall but slashes storage by nearly 98%. This significantly lowers infrastructure expenses, especially for organizations running RAG systems with tens of millions of chunks.

The practical takeaway

Lower storage requirements open the door to cheaper, more scalable vector search at the cost of adding seconds per query due to recomputing embeddings on the fly. This may be a reasonable trade-off for many who can tolerate slightly slower response times in exchange for much smaller, more manageable storage footprints. Builders juggling massive datasets and tight budgets should test LEANN as a way to cut cloud costs without sacrificing much accuracy. It also offers an open-source foundation to experiment with graph pruning approaches in vector search.

What to watch next

Performance improvements in embedding models could reduce query latency, making LEANN’s trade-off more attractive. Additional real-world benchmarks across diverse RAG workloads will clarify its operational limits. Look for integration efforts that make LEANN easier to adopt in popular vector databases or retrieval frameworks. Monitoring adoption by startups and AI operators balancing cost and performance will reveal if this approach reshapes infrastructure economics for large-scale vector search.

AI Quick Briefs Editorial Desk