Depression is a common mental health problem that affects millions of people around the world. Clinical interviews are a commonly used method to evaluate depression, but their dependence on professional clinical psychologists underscores the necessity for an automatic evaluation system. However, existing methods lack the use of multimodal complementary information and cannot fully capture complex correlations and mapping relationships between different modalities. Moreover, existing methods also fail to make full use of the complex semantic dependencies between utterances. To address these challenges, we propose a novel multimodal depression estimation framework named MultiDepNet, which is based on graph neural networks and attention mechanisms. The framework captures cross-modal mapping relationships between text, audio, and visual cues through a Multimodal Relation Capture Network (MRCN) based on bidirectional cross-attention layers, effectively integrating depression cues. Meanwhile, we utilize Semantic Context Graph Network (SCGN) to complement the semantic dependencies between contextual utterances and design weighting factor \(\alpha \) to adjust the weighting of different information in the model. We conducted extensive experiments on the DAIC-WOZ dataset, demonstrating that our method achieves significant improvements, reducing the MAE and RMSE by 5.01% and 11.99%, compared to previous methods. These results highlight the superior performance of our approach and validate its effectiveness.