Sentences in student’s essay and comments were analysed to examine the feasibility of evaluating ability of critical thinking disposition (CTD) by measuring this ability based on texts in a university class. Two approaches, such as term dependency in texts and automated essay scoring using a machine learning technique (BERT), were employed to predict writing performance and CTD factor scores. The results, which are based on the term dependency reflect the levels of essay scores and factor scores, and illustrate the relationships between dependency and ability level using corresponding analysis of these frequencies. A machine learning based approach can predict accurately essay and comment scores from these texts, however, the prediction for CTD factor scores needs further improvements.