Ship fuel consumption prediction is a crucial basis for ship navigation optimization. To enhance the accuracy of ship fuel consumption predictions, we proposed a prediction framework based on the fusion of channel and spatial information, which combined the long short-term memory network (LSTM) to capture temporal dynamic features with the convolutional neural network (CNN) feature extraction. Additionally, it incorporated the channel and spatial attention mechanism (CBAM) to improve feature representation and prediction performance. Compared to the benchmark LSTM network model, the convolutional neural memory network (Conv-LSTM-CBAM) with integrated attention mechanism increases the determination coefficient R2 of the prediction results by 11%, and reduces the root mean square error (RMSE) and mean absolute percentage error (MAPE) by 4.25% and 5.9%, respectively.