The application of ultrahigh definition endoscopy systems in minimally invasive surgeries has become increasingly widespread. However, their high resolution results in a reduced depth of field (DOF), making it difficult to achieve clear imaging across the entire frame. Unlike improvements in optical structures, we address this issue using a deep learning-based multi-focus image fusion (MFIF) approach. Traditional MFIF methods are less effective in endoscopic scenarios due to their inadequate design for extracting information from complex organ structures. To address these limitations, this work proposes a two-streamed cascaded encoder-decoder network that incorporates multi-scale feature extraction and fusion mechanisms validated in medical image segmentation. The network includes novel multi-scale fusion module with cross-axial attention that hierarchically integrates features using attention-guided weights and hybrid operations, effectively preserving intra-domain textures while modeling cross-domain dependencies. The framework is rigorously validated using novel real-world endoscopic datasets collected from imaging experimental platform. The experimental results demonstrate that the proposed method outperforms traditional approaches in benchmark tests. Code available at: https://github.com/luoyu5023/CTMFusion .