In response to the problems of strong subjectivity and low efficiency in traditional English oral automatic evaluation, decision tree algorithm was applied in the research to improve the accuracy and efficiency of evaluation through feature selection and performance optimization. English oral audio samples were collected and multidimensional scoring was conducted; the audio was converted into text format for denoising and volume standardization; speech features were extracted and fluency features were calculated; correlation analysis evaluated the importance of features and selected features with high correlation. The CART decision-tree model—was used to construct the model. Hyperparameter tuning and cross-validation methods were used to optimize the model, and the mean square error and processing time indicators were used to evaluate the effectiveness of the model. The research results indicated that the mean absolute error) of the decision tree model was only 0.015, and the average processing time was only 1.58 s. The method used improved the accuracy and efficiency of automatic English oral evaluation.