Accelerating Intra-frame Block Partitioning Decision for AVS3 Using Attention Mechanisms
摘要
The next-generation video coding standards, the Versatile Video Coding (VVC) and the Third Generation of Audio Video Coding Standard (AVS3), enhance intra-frame coding by introducing diverse block partitioning structures. The flexible structure allows for a more precise match to the texture of the Coding Units (CUs), but it also significantly increases the complexity of searching for optimal block partitioning results. This paper proposes a method for predicting the partitioning results at specific partitioning depth levels using attention mechanisms, in order to accelerate the search process. Specifically, we initially utilize a transformer encoder neural network to extract features from the Largest Coding Units (LCU). Following this, a transformer decoder sequentially predicts the partitioning of each CU in accordance with the order determined by the traditional recursive-tree. During the prediction process, the encoder-decoder attention mechanism within the transformer decoder efficiently extracts the relevant features of the current CU from the encoded LCU feature volume. Additionally, its masked self-attention mechanism could leverage historical decision results from its parent and neighboring CUs (i.e., the decoder token sequence) to enhance prediction accuracy. Moreover, we have designed CU information prompt tokens based on the predefined partitioning order to guide the decoder’s prediction process. By providing additional information, this approach reduces the learning difficulty and further improves accuracy. We applied the proposed method to accelerate AVS3 HPM-14.0. Experimental results show that the attention mechanism block partitioning prediction significantly reduces encoding time (62.30%), with only a 1.17% BDBR increase. This result outperforms previous acceleration work on AVS3 and is comparable to or even exceeds related work for VVC.