Offline handwritten signature verification has been a challenging pattern recognition problem due to high intra-writer variability and inter-class similarity. In this paper, we propose a multi-task interaction network (MTI) based on a cross-attention fusion mechanism, which can dynamically focus on the critical detail differences between input signature pairs and efficiently capture contrast cues by fusing the contextual information between two images. Additionally, we designed a multi-task learning framework that enables the model to verify the authenticity of a signature while enhancing its understanding of the signature’s writing style through recognition. By combining these two tasks, our network enhances overall verification accuracy by increasing the understanding of writing styles and distinguishing key detail differences between genuine and forged signatures through fine-grained comparison cues. Most current studies are based on signatures written in the same language script. Given the rich linguistic and cultural background in China’s Xinjiang region, we construct a multilingual offline signature dataset containing Uyghur, Kirghiz, Kazakh, and Chinese, the first comprehensive dataset combining character-based and letter-based signatures. The verification accuracies of our method on the publicly available datasets CEDAR, BHSig-H, BHSig-B, and our Multilingual dataset Mult-Sig reach 100%, 91.19%, 94.12%, and 92.88%, respectively, and extensive experiments demonstrate the effectiveness of the proposed method and its competitiveness with current state-of-the-art techniques.