Panoramic images and videos have become widely popular media formats. However, there are challenges in dealing with the lack of six degrees-of-freedom (6DOF) motion in panoramas. Recent advancements in novel view synthesis techniques have shown promising outcomes in indoor settings characterized by geometric structures. Nevertheless, the translation of these advancements to complex outdoor panorama, especially with single panorama as input, remains a formidable task. In this study, we propose a novel view synthesis pipeline that takes a single outdoor panorama as input. The method employs a dual-branch design that downsamples the input image to capture the global information of the complex outdoor scene and utilizes Multi-Sphere Images (MSI) for MSI-RGBA representation inference of the input panorama. To represent complex geometric shapes and multi-scale details, we introduce a high-resolution refinement branch to optimize the fine edges in the panorama, resulting in high-quality synthesized novel outdoor panorama. Our method has achieved significant performance improvements in single-image synthesis using the CARLA datasets, and it can be generalized to real outdoor panorama datasets. These endeavors contribute to advancing panoramic media towards a more comfortable immersive experience, ultimately enhancing the realism of immersive panoramic 6DOF roaming.