Semantic Search for Financial Products Using Embeddings and Pgvector
摘要
This article presents a semantic search system for financial products using vector embeddings and PostgreSQL with the pgvector extension. The all-MiniLM-L6-v2 model transforms attributes such as account type and currency into vector representations, facilitating searches based on semantic similarity. An automated pipeline was implemented that includes web scraping of data from the website of the Superintendency of Banking, Insurance, and Pension Fund Administrators (SBS) of Peru, followed by processing with Django and a frontend in React. The results show excellent performance in single-criterion queries, obtaining a Recall@10 close to 1.0, which indicates that the relevant results are among the top ten. In queries with combined criteria, such as entity plus location, Recall@10 reaches 0.5. It is concluded that the automation of data collection and processing, together with the generation of semantic embeddings and their efficient storage in PostgreSQL using pgvector, allowed the development of a system capable of interpreting queries in natural language and surpassing traditional methods, demonstrating high performance in tests with real data.