<p>Recently, the approach of extracting features of essays at different levels for joint learning and scoring using hybrid models has achieved excellent results. However, there are still some issues that need to be improved: (1) Splitting the essays into sentences of the same length when extracting sentence-level features may result in the truncation of sentence-level semantics; (2) multi-level representations of essays obtained through pre-trained language models (PLMs) are not sufficiently comprehensive and in depth to provide limited improvement in model performance; and (3) in extracting features from multi-level essay representations, finer-grained local features in sentences and dependencies among features are ignored. To address these issues, we propose an improved method for joint learning of multi-level features based on PLMs. Specifically, we first split the essays according to different semantic scales through a text splitter to maximize the retention of semantic and structural features at sentence level; secondly, we integrate the essay representations derived from the PLM and mine more comprehensive and deeper multi-level features from it; and finally, we aggregate sentence-level features and construct dependencies among them by a feature extraction fusion method. The value of average quadratic weighted kappa (QWK) on the Kaggle ASAP competition dataset is 0.809, which validates the effectiveness of our proposed method in the automated essay scoring (AES) task.</p>