Synthetic lethality (SL) is a genetic interaction in which the simultaneous perturbation of two genes leads to cell death, whereas perturbation of either gene alone is viable. Challenging in predicting synthetic lethality pairs usually comes from a highly biased source of data (only SL positive or only SL negative pairs) and insufficient data background (research data are usually single-omic). In this work, we create a large multi-omics dataset coming from diverse settings, including transcriptomics profiles, genetic perturbations (i.e., RNA interference, Clustered Regularly Interspaced Short Palindromic Repeats), and genomics data (i.e., nucleotide sequences and amino acid sequences). Here, we also propose a multimodal model containing different modules specifically designed to capture biological insights of each type of omic data. Quantitative results show that our method has a top-notch specificity in predicting synthetic lethality despite its simplicity compared to other advanced techniques like graph-based models with a specificity of 99.75%, a sensitivity of 90. 55% and precision of 97. 88%.