Source-free domain adaptation (SFDA) aims to address the challenge of model adaptation under target domain distribution shifts while being unable to access source domain data. Existing methods typically rely on pseudo-label generation or self-supervised tasks, which often leads to models being susceptible to noisy pseudo-labels. To overcome these limitations, this paper proposes a collaborative optimization framework CTO based on multimodal adapter drivers. The method introduces a vision-language multimodal large model (ViL) and constructs a style-semantic prompt bank to mitigate the complexity of fine-tuning. Furthermore, our design a cross-modal feature disentanglement mechanism to eliminate the impact of domain shifts and modality gaps on model generalization. Additionally, leveraging cross-modal transferability, our develop a novel pseudo-label generation strategy that integrates predictions from both text and iamge modalities to guide the model optimization process. Extensive experiments on four benchmark datasets demonstrate that CTO achieves superior performance compared to existing methods, validating its effectiveness.