Predicting the performance of students plays an essential role in education because it brings many benefits such as resource optimization, personalized learning, etc. International School - Vietnam National University, Hanoi, Vietnam has special training programs because of training in English; therefore, enrolled students must require English certificates. Students without certificates must enroll to study the pre-English courses. Preparing the resources for these students is very challenging; therefore, it is essential for the university to predict student performance accurately. In this paper, we propose an approach to predict the performance of students in pre-English courses by leveraging Large Language Models. By comparing to other predictive models, our approach outperforms significantly.