<p>The prevailing practices in the diagnosis of mental disorders mainly rely on the subjective assessments of patients and clinical interviews, which could also be very biased and thus unreliable. Automated detection of mental health disorders is still very difficult, and one of the reasons is that it has to incorporate data of multiple types like, for instance, the integration of speech, text, audio features, and videos of facial expressions. To provide the solution to the problem, a new approach of the multimodal deep learning framework is proposed that captures effectively the cross-modal interactions by combining data from text, audio, and visual inputs. In particular, a Multimodal Deep Denoising Autoencoder (Multi-DDAE) is the one that is utilized to merge and eliminate noise coming from the session-level audiovisual signals, and at the same time, the Paragraph Vector (PV) is the one that is used to convert the interview transcripts into document-level representations that are capable of capturing the linguistic patterns linked to mental illness. After that, the fused representations are passed to a Bidirectional Long Short-Term Memory (BLSTM) network that models them and that can grasp the temporal dependencies from both the past and the future contexts via a sliding window mechanism. Our methods have been tested with two clinically significant datasets, that is, the Bipolar Disorder Corpus (BDC) for the case of bipolar disorder and the Extended Distress Analysis Interview Corpus (E-DAIC) for the case of depression and psychological distress. The experimental outcomes not only indicate better performance than existing methods but also this superiority serves for the effectiveness of multimodal representation learning in the diagnosis of mental health issues.</p>