Setting up, running, and evaluating demand forecasts entails several unexpected pitfalls, which jeopardize the reliability of the forecasts and the interpretation of evaluation metrics. These challenges lead to sometimes necessary, sometimes harmful interventions such as manual forecast overrides. We review the causes and effects of common pitfalls in setting up and evaluating demand forecasts for supply chain applications. Overcoming these challenges is necessary for the adoption of modern AI/ML-based systems, since the full potential of forecasting systems can only be unleashed when users know how to set up a forecast, what to expect from it, how to rate it, and when and when not to intervene.