Federated split learning is confronted with the challenges of high communication costs and insufficient adaptability to varying network environments, which severely restrict the deployment efficiency of models in edge intelligence scenarios. To address these issues, we propose Reinforcement Learning-based Federated Split Learning (RLFSL), a novel federated split learning scheme that integrates multi-strategy communication optimization with adaptive model splitting. In federated split learning, the communication cost primarily stems from the forward and backward propagation between devices and the server. To mitigate this cost, our proposed scheme employs three strategies, namely pre-trained initialization, activation reuse, and quantization-based compression. Moreover, when the network bandwidth fluctuates, RLFSL utilizes a reinforcement learning algorithm to implement an adaptive model splitting mechanism that minimizes training latency. Efficient bandwidth assessment method based on partial weight communication is used to reduce the communication cost between the agent and the environment. Experiments conducted on the CIFAR-10 and CIFAR-100 datasets demonstrate that RLFSL can significantly improve model training efficiency and reduce communication costs, while achieving comparable model accuracy compared to baselines. Furthermore, the results of applying the trained reinforcement learning agent to different neural networks also show the generalization capability of RLFSL.