<p>Precise vessel trajectory prediction is of great importance to the maritime safety. However, most of the existing methods only utilize Automatic Identification System (AIS) information and ignore the impact of marine physical data. Moreover, combining point-based AIS data with coarse-resolution gridded marine information comes with its own set of spatiotemporal alignment challenges. These issues make it harder to capture the spatial heterogeneity of physical factors accurately. To address the above issues, we propose a vessel trajectory prediction method based on Transformer architecture and marine physical data, Distance-Modulated Transformer (DM-Trans). Firstly, we adopt a multi-dimensional fusion downsampling method to decrease the amount of trajectory data and align the heterogeneous data on a grid for consistency. Secondly, our defined Distance-Modulated CNN (DM-CNN) will modulate the weights of multi-channel physical features and extract features based on the precise relative position of trajectory points on the grid, which can accurately reflect the impact of local environment on trajectory. Finally, we integrate these physical features and trajectory data together and feed the result into Transformer for prediction. Our method is evaluated on real-world datasets and the extensive experiments show that compared with some baseline models, DM-Trans can significantly improve the prediction accuracy. The further ablation experiments also validate the effectiveness of different parts.</p>