A Novel Semantic Embedding Using Complex Mapping
摘要
An inseparable part of natural language processing (NLP) tasks is word embedding, which bridges machine and human language understanding. Although embedding as a representation system revolutionized NLP profoundly, an investigation of recently published word representation models leads us to focus on its limitations. The essence of word embedding is not fully developed to correctly decode contextual words in longer texts or handle phrases and homonyms, even heteronyms and polysemes. This issue, in turn, increases the dependence on various neural networks, more computations, and the cost of solving ambiguity. In order to overcome this issue, an approach suited to encode the inter-dependency among embeddings is presented. This approach is premised on the superposition principle, which helps the systems process sentences differently for the first time ever. Superposition adaptation is implemented by proposing a complex mapping model to both prevent interference of meaning and account for words’ relations in the early phase of feeding word embedding to models. The proposed method successfully decodes meaningful correlations between words. It is evaluated on three NLP tasks with three public datasets in two languages (Chinese and English). Detailed experiments demonstrate that the proposed approach outperforms state-of-the-art methods, achieving 17% error reduction on average. Further discussion centers on results to confirm the effectiveness and reusability of the method for other tasks as well.