<p>Machine learning predictive models rely on data to make predictions for new input data. However, accurate predictions are not always the end goal; practitioners often aim to make informed decisions through optimization problems (OPs) based on these predictions. While the idea that better predictions lead to better decisions was widely accepted, the latest literature highlights that even small inaccuracies in predictions can lead to poor decisions depending on the structure of the OP. Therefore, recent research has been focused on end-to-end learning approaches that directly improve decision quality without considering prediction accuracy when solving data-driven OPs. Some of these end-to-end learning approaches are mainly called “predict-and-optimize” (PaO), and they aim to learn a predictor based on the quality of the downstream task decisions by incorporating mathematical programming into the learning process. This literature review discusses the variations of and approaches to PaO problems by proposing a unified notation and a taxonomy for them. Throughout the paper, we aim to provide a valuable roadmap for researchers and practitioners in the field, guiding them to choose data-driven methods to solve their decision problems effectively.</p>