Memory-Constrained DiskANN: Efficient Approximate Nearest Neighbor Search Under Resource Constraints
摘要
Approximate nearest neighbor search is a critical component in large-scale high-dimensional retrieval systems. This paper addresses the 2025 SISAP Indexing Challenge task 1, which requires searching 23 million 384-dimensional vectors under tight memory and disk constraints, with an average recall threshold of 0.70. To meet these constraints, we introduce two optimizations to the DiskANN pipeline. First, we apply PCA-based dimensionality reduction to ensure memory usage stays within bounds. Second, we refine the second-assignment mechanism by sorting points based on distance and assigning only the closer half to the secondary shard. Our approach achieves approximately 0.80 recall while adhering to the challenge’s resource and performance limitations.