<p>Recent action quality assessment methods attempt to decompose the entire video into sub-actions of the same level, which still suffers from information redundancy and lacks certain interpretability. Through studying and analysing the evaluation rules of relevant sports events, we assume that certain critical moments in the actions will significantly impact the assessment of the entire action, if the weight of these critical moments in the evaluation process can be increased, then the results can be more accurate and more persuasive. To validate this idea, we propose a moment-aware approach for action quality assessment, a novel moment-aware module is used to explore the key moments in action globally and locally to obtain more reliable evaluation results. Extensive experiments have shown that our module captures key moments that are close to the focus of human referees, and our method has also achieved favorable results on multiple public AQA benchmarks, the Spearman’s rank correlation reached 0.9360 on FineDiving dataset and 0.8797 on AQA-7 dataset.</p>