The sperm cell is recognized nowadays as a key epigenetic vector, transmitting not only the paternal genome but also a complex epigenetic content—including DNA methylation, chromatin code and structure, and non-coding RNAs—to the zygote. This epigenome is highly sensitive to environmental factors, with paternal exposures such as dietary intake, lifestyle choices, stress, chemical hazards, pollution, or reprotoxic treatments (among others) influencing its programming. Genetic factors also modulate the sperm epigenetic cargo, supporting that a combination of complex multifactorial cues will define the final epigenome before conception. Alterations on the sperm epigenetic alterations, apart from having the potential to be used as a diagnostic tool in reproductive medicine, are central to the paradigm of Paternal Origins of Health and Disease (POHaD), linking sperm epigenetic alterations not only with fertility problems, but also with the transmission of altered traits to the zygote, potentially impacting embryo and placenta development during pregnancy, associated with a higher incidence of maternal and offspring long-term health problems. This chapter reviews recent evidence on how environmental and genetic perturbations modify the sperm epigenome, highlighting its plasticity and the challenges posed by the heterogeneity among studies. We discuss the interplay between exposome and genotype, and the need for further research to clarify the mechanisms of epigenetic inheritance and its long-term consequences.