In the field of criminal justice, artificial intelligence (AI) models have emerged as valuable tools for automating the identification of relevant legal statutes, whereby the models predict which statutes in the judicial system are relevant to the facts of a case. However, the opacity of these models presents a challenge, especially when interpretability and transparency are paramount. This paper introduces models that combine long-context encoder models with explainability techniques to achieve both predictive accuracy and interpretability in statute identification. By employing long-context encoders, the proposed architecture processes extended legal documents in a straightforward, end-to-end manner, offering a simplified and efficient alternative to more complex hierarchical models. To ensure transparency in predictions, we incorporate PartitionExplainer, an explainability method that approximates the contribution of grouped features to model predictions. This increases the efficiency of generating explanations, as it would compute Shapley values in polynomial instead of exponential time.