Understanding LLM Entity Representations for Efficient and Interpretable Entity Detection
摘要
Named Entities are fundamental to understand and process natural language, yet mention detection—identifying which text spans refer to entities—remains a primary performance bottleneck. Current approaches require supervised training for each schema, while generalist extractors like GLiNER need schema specification at inference time. Recent mechanistic interpretability research suggests that LLMs may naturally learn to encode entity structures during pretraining through mechanisms like binding IDs. This motivates our central hypothesis: if LLMs naturally develop entity detection capabilities, we should be able to extract them efficiently without task-specific training or schema specification. This PhD research investigates four main questions: how LLMs represent multi-token entities, what mechanisms enable entity manipulation and relational reasoning, whether we can efficiently extract mentions from their hidden representations and how can these results finally enhance information retrieval. We propose using task vectors for entity mention reconstruction, and lightweight detection architectures for generalist mention detection. This work aims to enable interpretable entity processing, efficient extraction with lightweight models, and zero-shot transfer across datasets.