Accurate segmentation and classification of hepatic veins and portal veins in abdominal magnetic resonance images are essential to surgical planning of liver tumor ablation and resection. These remain challenging due to the complex vascular structure and the close intensity of hepatic and portal veins in enhanced magnetic resonance images. This work proposes a new small deep-learning model of hybrid attention-residual networks for precise extraction of hepatic and portal veins. Specifically, the encoder of our model uses residual units to extract abundant local features and employs recursive gated convolution to extract global spatial dependence through high-order feature interactions. The decoder uses cross-attention to fuse local texture information and global semantic information to segment a more complete vascular structure and distinguish hepatic and portal veins more accurately. We trained and evaluated our model on collected clinical abdominal magnetic resonance images, with the experimental results demonstrating that our method works better than the other segmentation approaches, attaining much higher dice similarity coefficient 79.60% and 78.27% in hepatic veins and portal veins segmentation, which are superior to 74.53% and 76.07% of state-of-the-art deep learning segmentation models.