A Variance-Invariance Approach to Better Word Embeddings
摘要
Word embeddings continue to be used in natural language processing for their efficiency, interpretability, and ease of integration into a wide range of applications. While effective at capturing distributional semantics from large unstructured corpora, they often miss fine-grained lexical relationships such as synonymy, antonymy, and hierarchy. This paper introduces a lightweight post-processing framework designed to enrich pre-trained word embeddings by incorporating structured lexical knowledge from resources like WordNet. Our approach leverages a self-supervised training objective based on variance and invariance principles to adjust word vector relationships according to their semantic roles. This enriched structure is then propagated back to the original embedding space without requiring retraining. Experimental results show that our method consistently improves word similarity performance by an average of 8%, demonstrating the value of integrating external lexical knowledge into pre-trained embeddings for enhanced semantic representation.