The growing volume of electronic health records (EHRs) has made deep learning a vital tool for real-time treatment analysis. While early models focused on single-modality learning, their limited informational perspective constrained performance. Recently, multimodal approaches that integrate complementary data sources have shown performance gains. However, two major challenges persist: 1) A large amount of unlabeled data remains underutilized. Unlabeled medical data is prevalent due to privacy concerns and other factors. Most end-to-end models rely on labeled data and thus fail to exploit the full potential of available data sources. 2) Insufficient case-level semantic constraints in existing pretraining methods. Although a few pretraining models enable the utilization of unlabeled data, current methods primarily establish constraints at the modality level, neglecting the constraints on holistic representations. This limitation undermines their effectiveness in downstream clinical tasks. To fill this gap, we propose a dual-level contrastive semi-supervised representation learning framework that leverages both hourly-level and case-level constraints to utilize unlabeled data. First, a time-aware alignment module aligns different modalities and introduces fine-grained cross-modal contrastive learning at the hourly level, enhancing local consistency across modalities. Then, to generate case-level positive samples, we introduce a memory-based case-level augmentation strategy. We combine partially dropped current patient representations with those of other patients retrieved from the memory bank, forming case-level positive pairs with hidden state augmentation. By jointly optimizing representations at the local temporal scale and the global case level, our model generates more comprehensive representations. Experiments on real-world tasks demonstrate our method achieves state-of-the-art performance and maintains representation quality under sparse labeling conditions.