Word embeddings continue to be used in natural language processing for their efficiency, interpretability, and ease of integration into a wide range of applications. While effective at capturing distributional semantics from large unstructured corpora, they often miss fine-grained lexical relationships such as synonymy, antonymy, and hierarchy. This paper introduces a lightweight post-processing framework designed to enrich pre-trained word embeddings by incorporating structured lexical knowledge from resources like WordNet. Our approach leverages a self-supervised training objective based on variance and invariance principles to adjust word vector relationships according to their semantic roles. This enriched structure is then propagated back to the original embedding space without requiring retraining. Experimental results show that our method consistently improves word similarity performance by an average of 8%, demonstrating the value of integrating external lexical knowledge into pre-trained embeddings for enhanced semantic representation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Variance-Invariance Approach to Better Word Embeddings

  • Zainab Albujasim,
  • Diana Inkpen

摘要

Word embeddings continue to be used in natural language processing for their efficiency, interpretability, and ease of integration into a wide range of applications. While effective at capturing distributional semantics from large unstructured corpora, they often miss fine-grained lexical relationships such as synonymy, antonymy, and hierarchy. This paper introduces a lightweight post-processing framework designed to enrich pre-trained word embeddings by incorporating structured lexical knowledge from resources like WordNet. Our approach leverages a self-supervised training objective based on variance and invariance principles to adjust word vector relationships according to their semantic roles. This enriched structure is then propagated back to the original embedding space without requiring retraining. Experimental results show that our method consistently improves word similarity performance by an average of 8%, demonstrating the value of integrating external lexical knowledge into pre-trained embeddings for enhanced semantic representation.