FCFormer: Fourier Convolution Vision Transformer for Image Classification
摘要
In recent years, Vision Transformers have emerged as a prominent model in the field of image classification, owing to their superior ability to model global contextual information and their impressive performance across various tasks. However, Vision Transformers generally suffer from high computational costs, primarily due to the quadratic complexity of the self-attention mechanism, which leads to time-consuming inference and insufficient local feature learning. To address these challenges, this paper proposes a novel Fourier Convolutional Vision Transformer architecture, termed FCFormer. FCFormer enhances representation learning by combining frequency-domain feature extraction with deep integration of multi-level convolution and self-attention. Specifically, the architecture leverages a Fourier frequency-domain processing module to efficiently extract rich frequency features at a low computational cost while highlighting critical regions for representation learning. Additionally, the integrated aggregated perceptual self-attention mechanism effectively reduces computational overhead while preserving the model’s ability to selectively focus on key parts of image data. Furthermore, a locally enhanced gated linear unit is introduced to improve local feature perception and enable multi-dimensional feature aggregation. Experimental results demonstrate that FCFormer achieves a better balance between parameter efficiency and classification accuracy compared to classical CNN and Transformer models in image classification tasks..