In medical imaging, especially CT scanning, accurate classification of arteries and veins is crucial for diagnosis and treatment. Existing deep learning methods, although capable of capturing arterial and venous features, often process voxel points independently and lack a holistic understanding of vascular branch structures. This limitation results in misclassifications at branch points, especially in distal branches, necessitating post-processing for correction. Traditional post-processing algorithms, such as arterial and venous density calculations or graph cut methods, can correct short intrusions but are limited in addressing long intrusions or mixed intrusions, hindering the application in complex vascular networks. To address this issue, this paper proposes V-GnNet, a fusion learning method combining voxel-based predictions and graph structure node predictions. Firstly, V-GnNet employs an iterative 3D neural network based on the UNet architecture, IterUNet3D, for preliminary classification of vascular data. IterUNet3D enhances the classification performance of arteries and veins by enriching the network's multi-level feature inputs through iterative mini-UNet3D modules. Subsequently, a special graph structure is established by extracting the vascular skeleton, integrating priori knowledge into a feature matrix, and utilizing a Graph Attention Network (GAT) for node classification of the IterUNet3D results. Finally, a voting algorithm fuses the voxel prediction results and node prediction results, ensuring consistent branch classification in artery and vein separation, therefore addressing the challenges of branch misclassification. Experimental results demonstrate that V-GnNet significantly improves the accuracy and consistency of pulmonary artery and vein classification, effectively reducing branch misjudgments and mutual intrusions, which showcases its great potential in medical image processing.