Medical image segmentation is crucial for supporting clinicians in lesion regions locating, disease diagnosis and treatment planning. In recent years, the Transformer architecture has become more and more popular for medical image segmentation due to its outstanding ability in modeling long-range dependencies. Many methods combine CNN and Transformer to overcome the limitations of Transformer in local feature extraction. However, most of these methods fail to effectively utilize both CNN and Transformer to fully extract local and global semantic information. To address this issue, we propose a novel Hybrid Encoder and Decoder Network (HED-Net) for medical image segmentation, which effectively extracts local and global features by constructing a hybrid CNN-Transformer encoder and decoder. In the encoder, a mixed encoder structure is built by alternately using CNN and Transformer to extract local and global semantic information. In the decoder, a Parallel Feature Extraction Module (PFEM) is designed to combine CNN and Transformer in parallel, generating hierarchical representations while fusing low-level and high-level features to enhance segmentation performance. Experimental results show that HED-Net achieves the highest Dice score of 81.78% and 90.01% on the Synapse and ACDC datasets, significantly outperforms most state-of-the-art medical image segmentation networks.