Legal text classification is a critical task in automating legal information retrieval and decision-making, yet it faces persistent challenges due to the complexity and length of legal documents. Traditional lightweight models, such as LSTM, offer computational efficiency but lack attention mechanisms, which limit their performance on long and context-rich legal texts. This study explores the implementation of DistilBERT, a distilled transformer model, to enhance classification accuracy while maintaining efficiency. Two benchmark datasets, the Terms of Service (ToS) dataset and 57k English EU legislative documents (EURLEX57k), were used to evaluate model performance on short and long legal documents, respectively. Experimental results show that DistilBERT significantly outperforms the LegaLMFiT baseline on short-text classification, demonstrating superior contextual understanding. However, it underperforms on long-text multi-label classification tasks, highlighting that architecture suitability remains task-dependent. The findings underscore the importance of striking a balance between model design, attention mechanisms, and pretraining strategies to address the complexities of the legal domain within computational constraints.