RAG Embeddings Storage Optimization Through Quantization and Dimensionality Reduction
摘要
Retrieval-Augmented Generation is a powerful framework that enhances language models by retrieving relevant information from large external knowledge bases, relying heavily on high-dimensional vector embeddings. However, storing these embeddings at scale poses significant challenges due to their substantial memory requirements. To address this, we investigate two complementary strategies: quantization of embeddings using low-bit floating point formats, and dimensionality reduction via Principal Component Analysis (PCA). Our experiments, measuring performance using \(nDCG@10\) (the standard MTEB Retrieval benchmark metric for ranking quality) relative to the float32 baseline, show that low-bit formats like float8 achieve 4x storage reduction with minimal (<0.3%) performance loss, significantly outperforming int8 at the same compression level. Notably, combining float8 with moderate PCA (e.g., retaining 50% dimensions) yields even better trade-offs, achieving 8x total compression with less performance degradation than int8 alone (e.g., a 0.62% drop vs 1.53% for nomic-embed-text-v1.5) while requiring only half the storage space. These findings highlight the effectiveness of combining float8 quantization and dimensionality reduction for efficient RAG embedding storage with minimal impact on retrieval quality.