Cyber attack technology is being upgraded at an unprecedented speed, making the field of network security face unprecedented complex challenges. It is necessary to comprehensively upgrade cyber attack detection technology to cope with the current security situation. By deeply studying the practical performance of traditional machine learning algorithms, we propose a novel detection model based on clustering and adaptive model selection. By using data clustering, the complex network data is divided into multiple subsets with similar characteristics. According to the data characteristics of different subsets, the most suitable mode can be chosen by the adaptive model selection, which can give full play to the advantages of each algorithm. Meanwhile, a category balance processing mechanism is added during the process, which effectively solved the problem of data imbalance and improved the detection ability of the model for attack samples. The new model architecture breaks through the shortcomings of traditional algorithms in dealing with complex network data features and data category imbalance. Experimental results show that the new model has an effect on improving Accuracy, Precision and F1 score, and is significant for enhancing network security.