Artificial Neural Networks have demonstrated the versatility to adapt to different problems while at the same time achieving high predictive performance. A characteristic of these models is their high-computation demands, which translates into high energy consumption. Spiking Neural Networks (SNN) have been proposed as an energy-efficient alternative with promising results in several tasks. Despite being a promising alternative that could motivate massive deployment, research on the interpretation of these models is almost non-existent. To address this problem, we propose SVEBI, a method that extracts insights on the representation encoded by SNNs, via the analysis of sparse relevant internal units that drive the decision-making process. In addition, we show the use of the relevant units as a means to justify, i.e. explain, the predictions made by an SNN for a given input. In this explanation/justification task, our experiments show SVEBI is comparable, if not superior, to existing methods.