As large language models (LLMs) scale to unprecedented sizes, traditional distributed training in homogeneous cluster environments increasingly falls short due to resource constraints and prohibitive computational costs. The integration of cross-regional distributed computing infrastructures for distributed training has become a current research focus. However, existing distributed training methods in heterogeneous environments mainly focus on a single heterogeneous feature, making it challenging to adapt effectively to a dual heterogeneous environment where computing and communication are complexly intertwined. Challenges such as imbalanced training costs and sensitivity to communication delays have significantly hindered the efficiency of distributed training in heterogeneous environments. To address these challenges, we propose a stable hybrid parallel distributed training architecture in dual-heterogeneous environments, SHPTA, which integrates heterogeneous devices across multiple regions to ensure stable and efficient distributed training of LLMs. From a global perspective, we introduce a HPO algorithm that comprehensively considers factors such as different bandwidths, delays, device performance, and quantity, accurately evaluates the parallel cost of the pipeline stage, and determines the optimal training device combination. Following this, the HAS mechanism is introduced from a fine-grained perspective during the training process. By dynamically reconstructing the communication topology structure, the optimal data parallel communication group is quickly formed, effectively reducing the synchronization overhead caused by device delays. Experimental results demonstrate that, compared to existing state-of-the-art methods, our method SHPTA reduces training time by 17.4% and achieves throughput increase of 26.8%.