Enhancing Prompt Tuning with Multitask Embedding Reparameterization
摘要
Prompt tuning has emerged as a highly effective technique in the field of natural language processing, particularly for adapting pre-trained language models to downstream tasks. Nevertheless, conventional prompt tuning techniques frequently encounter difficulties in effectively leveraging knowledge across multiple tasks, which can result in the isolation of knowledge and suboptimal performance in multitask learning scenarios. In this paper, we propose Multitask Embedding Reparameterization (MER), a novel approach designed to enhance prompt tuning by facilitating robust cross-task knowledge transfer. MER achieves this by reparameterizing task-specific prompt embeddings through a shared fusion module, enabling the model to capture and integrate knowledge from multiple tasks. This approach not only enhances model generalization but also diminishes the overall parameter footprint in comparison to conventional fine-tuning methodologies. Extensive empirical evaluations on the SuperGLUE benchmark demonstrate that MER consistently outperforms existing prompt tuning techniques, achieving substantial improvements in cross-task knowledge transfer and overall performance while maintaining high parameter efficiency. The results demonstrate the potential of MER as a promising solution for parameter-efficient and scalable multitask learning in a range of NLP applications.