Click-Through Rate prediction, which aims to estimate the probability of a user clicking on an item, is a key task in online advertising. Existing CTR models concentrate on modeling the feature interactions within a single domain, thereby rendering them inadequate for fulfilling the requisites of multi-domain scenarios. Some recent approaches propose intricate architectures to enhance knowledge sharing across multiple domains. However, they encounter difficulties when being transferred to new domains, owing to their reliance on the modeling of ID features. To tackle this issue, we propose the Universal Feature Interaction Network (UFIN) approach for CTR prediction. UFIN exploits the textual data to learn the universal feature interactions that can be effectively transferred across diverse domains. Specifically, we regard the text and feature as two different modalities and develop an encoder-decoder network to enforce the transference of data from text modality to feature modality. Building upon the above foundation, we devise a mixture-of-experts enhanced adaptive interaction model to learn the transferable collaborative patterns across multiple domains. As such, UFIN can effectively bridge the semantic gap to learn the common knowledge across various domains, surpassing the constraints of ID-based models. Extensive experiments conducted on eight datasets show the effectiveness of UFIN. Our code is available at https://github.com/RUCAIBox/UFIN .