Linguistics has often relied on written language or on transcripts of oral data to make claims about language, overlooking signed language (SL) research and the multimodal dimension of spoken languages (SpL). This situation has resulted in a biased and incomplete understanding of language. In this paper, we report on the building and use of two directly comparable datasets of a SL and SpL, namely the LSFB Corpus and the FRAPé Corpus. After introducing the datasets separately in their respective research contexts, we present the theoretical significance of collecting and using directly comparable corpora of SpLs and SLs. We then highlight the extent to which directly comparable corpora can help overcome challenges in (SL) linguistics and gesture studies. With this aim in mind, we present comparative work conducted at the LSFB-Lab investigating the use of depiction, reformulation, prosody, and interaction management in LSFB and Belgian French. Finally, we point to potential cross-linguistic research avenues offered by corpora that capture language use in a directly comparable way.