The precise estimation of seismic arrival times, commonly referred to as first-break picking, is a critical problem in seismic research due to its important role in various seismological applications such as statics correction processing. In recent years, there have been several deep learning algorithms designed specifically for 2D seismic arrival time picking. A widely used approach is to treat the 2D arrival picking problem as a 2D image segmentation problem and employ a deep semantic segmentation model for end-to-end first break picking. However, the first break mask generated from this method often fails to meet the uniqueness of first arrival time according to certain noises. In order to alleviate this problem, we propose a difference-enhanced learning method of the deep semantic segmentation network for the first break picking problem by designing a new kind of loss function, which actually improves the quality of mask generation and arrival time accuracy. It is demonstrated by extensive experiments on a real seismic dataset that our proposed difference-enhanced learning method is effective and outperforms the conventional learning methods for deep semantic segmentation models on the estimation of seismic arrival times.