Flow field prediction is essential in computer vision, particularly in engineering and scientific computing. Accurate flow field prediction is crucial for enhancing aerodynamics, reducing energy consumption, and improving safety in automotive design. However, traditional methods, such as computational fluid dynamics (CFD) face challenges due to high computational complexity, hindering real-time prediction and design optimization. To tackle these challenges, we introduce SwinPINet, an attention-driven neural network that incorporates physical insights. SwinPINet establishes a direct connection between the shape and flow field on a Cartesian grid, streamlining input demands in contrast to many current methods. Additionally, we designed an upsampling decoder specifically for flow field prediction, which eliminates checkerboard artifacts. Experimental results demonstrate that SwinPINet achieves high accuracy in predicting the flow field around vehicles, outperforming existing models. SwinPINet also exhibits excellent generalization ability and effectively predicts the velocity and pressure fields of two-dimensional airfoils. SwinPINet is positioned to serve as an efficient tool for evaluating aerodynamic performance, with promising potential to accelerate the design and optimization of aerodynamic systems.