As modern software-intensive systems grow in complexity and autonomy there is a demand for them to incorporate services that explain their actions. Such explanation services allow stakeholders, especially end-users, to develop trust that the corresponding system complies with its objectives and limitations. Before such explanation services can be designed, their requirements need to be analyzed, in terms of both the kinds of stakeholder questions such services can entertain and the content of the answers they respond with. We propose a framework for goal-oriented modeling and analysis of explanation requirements. Explanation requirements are captured as stakeholder goals and are subsequently analyzed into specific explanation tasks and explanation interaction templates. By further extending the models with temporal and causal constraints, the resulting conceptual model can be used to generate answers to certain families of stakeholder questions. Formalization of the model allows both simulations of the explanation service and, under restrictive assumptions, a working prototype thereof.