ArtemisSearch: A Multimodal Search Engine for Efficient Video Log-Life Event Retrieval Using Time-Segmented Queries and Vision Transformer-Based Feature Extraction
摘要
In this century, search engines have emerged as a crucial component of the technological landscape. Enterprises require a search engine to retrieve specific information within a particular field. However, they face various challenges due to the rapidly increasing volume of data and the need for effective database management to handle diverse data types. Additionally, the search for data is hindered by difficulties in matching queries with key frames or the limitations in understanding query context. In this paper, we introduce ArtemisSearch, a text-based multimodal search engine designed for temporal event retrieval in videos. In the proposed system, an efficient algorithm for Content-Based Image Retrieval (CBIR) using ViT-H/14 and BEiT3 for feature extraction and an open-source vector database, Milvus, our system efficiently retrieves events by leveraging temporal segmentation of queries and matching embeddings for Artificial Intelligence (AI) applications. Additionally, we developed a web application that allows end users to easily create temporally-aware descriptive queries, efficiently explore top results, and view precise video previews at relevant timestamps. The ArtemisSearch method represents a significant advancement in temporal video retrieval, with potential applications across diverse fields, leading to a smoother and more accurate video search experience.