With the rapid development of Internet of Vehicles (IoV) technology, the security of vehicle networks has become a critical concern. This paper presents a deep learning-based security protection model for IoV, aimed at efficiently detecting and mitigating network attacks. The model pretrains both the BiLSTM network and the CNN network, and then feeds the pretrained features into a cross-attention mechanism for multimodal feature fusion. This approach enhances the model’s ability to recognize complex attack patterns. To evaluate the model’s performance, we conducted experiments on the publicly available Car-Hacking and CAN-FD datasets. The results show that the model achieved 99.99% accuracy and 99.93% F1-score on the Car-Hacking dataset, and 99.99% accuracy and 99.98% F1-score on the CAN-FD dataset. Furthermore, when trained on the CAN-FD dataset and tested on the Car-Hacking dataset for Denial of Service (DoS) attacks, the model achieved 95.8% accuracy. These results demonstrate that the proposed BMF-Net model is highly effective at detecting various attack types in IoV, with strong generalization capabilities across different datasets, providing a robust solution for IoV security.