Optical Flow Estimation Using Transformer-Based Deep Neural Networks
摘要
In this study, an optical flow estimation approach using transformer-based deep neural networks is proposed, which consists of spatial feature extraction module, recurrent feature extraction module, matching feature module, coarse matching module, context feature encoding module, context attention module, and recurrent module. Two feature extraction modules extract spatiotemporal features from video frames, matching feature module enhances extracted spatiotemporal features for computing pixelwise correspondences, coarse matching module estimates the initial optical flow, and context feature encoding module and context attention module extract object-level features and propagate motion features to occluded regions, respectively. Finally, recurrent module performs iterative optical flow refinement and generates the final optical flow by upsampling. Based on the experimental results obtained in this study, in terms of three objective performance metrics and subjective evaluation, the performance of the proposed approach is better than those of six comparison approaches.