Event Detection from News Articles Using Lexical and Contextual Ranking Models in IR Systems
摘要
In the era of IR, event detection has moved beyond simple keyword searches to utilize advanced techniques to extract relevant events from massive news article datasets. The rapid growth of news highlights the need for efficient information retrieval techniques to capture the most relevant events. Traditional lexical-based retrieval methods, such as Whoosh and BM25, are effective in keyword matching; however, they have some limitations in understanding the semantic events from the indexed text. To enhance this limitation, this study introduces a Transformer-based deep learning model for Natural Language Processing (NLP), such as BERT, capable of capturing contextual relationships and improving the relevance of data. This research also explores an optimized approach that seamlessly integrates Whoosh for efficient indexing, BM25 for probabilistic ranking, and BERT for neural re-ranking, designed to improve event detection performance. Additionally, Named Entity Recognition (NER) significantly enhances event extraction by accurately identifying real-world entities like individuals, locations, organizations. The results of this research indicate that the integration of lexical models (Whoosh and BM25) with neural ranking models (BERT) significantly enhances precision, recall, and relevance, thereby exceeding the performance of traditional retrieval techniques. In our experiments BERT achieved a relevance score of 62%, outperforming BM25, which scored 55%. This demonstrates superior ability to capture contextual and semantic relationship in text. In conclusion, this study articulates prospective directions for future research within the realm of event detection, improving the efficacy of information retrieval in rapidly evolving news environments.