Vector Database Benchmarking: A Face Retrieval Use Case
摘要
Recently, several database management systems shave been developed to support the efficient handling of high-dimensional vector data and fast and accurate similarity searches. These systems differ significantly in their overall architecture and implementation of indexing and querying techniques, which may impact their performance in real-world scenarios. However, no official performance comparison exists to guide users in selecting the right system. We present a controlled experiment evaluating six widely used vector database management systems in a real-world face retrieval scenario to fill this gap. Although the use case involves face retrieval, the results are broadly applicable, as the core evaluation focuses on database performance over vector embeddings, regardless of their origin. The scenario utilises a dataset of 1TB of images from which a total of 1,730,000 face vector embeddings were identified and extracted; the extraction was performed using two different vectorisation techniques, resulting in vectors of two different dimensions, i.e., sizes. The evaluation is conducted for both vector sizes on the dataset divided into small, medium, and large subsets to investigate the impact of different data processing loads on system performance. The evaluated performance aspects include insertion and deletion times, initialisation overhead, and search latency. In addition to performance metrics, face retrieval quality is assessed using precision, F1, and specificity.