Tabamba: a Mamba-Transformer hybrid model for table structure recognition
摘要
Table structure recognition (TSR) is the process of converting the visual layout and logical relationships within table images into a machine-readable format. Following the impressive success of Transformers in various generation tasks, the image-to-sequence approach utilizing Transformer architecture has become a dominant paradigm in TSR. However, the self-attention mechanism fundamental to Transformers introduces quadratic computational complexity, making inference computationally expensive for long sequences. To address this efficiency bottleneck, we propose Tabamba, a novel Mamba-Transformer hybrid model for table structure recognition. Mamba is an attention-free architecture that achieves linear-time sequence modeling complexity. Tabamba adopts an encoder-decoder paradigm: a visual encoder employs a four-directional-scanning Mamba (4D-Mamba) layer to encode rich contextual features from the input table images. The architecture then utilizes a structure decoder, composed of four Mamba-Transformer hybrid decoder layers, to generate an HTML-based sequence that represents the logical table structure. This is followed by a bounding box decoder, which uses two hybrid layers to regress the spatial coordinates of non-empty cells. To the best of our knowledge, this is the first attempt to apply Mamba to table structure recognition. Extensive experiments conducted on two established benchmarks, PubTabNet and FinTabNet, demonstrate that our model achieves competitive results in both logical and physical structure recognition while maintaining a much smaller model size and higher inference speed compared to the state-of-art methods.