The integration of deep learning into medical vision applications has led to a growing demand for interpretable predictions. Typically, classification and explainability are treated as separate processes, with explainability methods applied post hoc to pre-trained classifiers. However, this decoupling introduces additional computational costs and may lead to explanations misaligned with the underlying model. In this paper, we propose One For All (OFA), an efficient, single-stage approach that jointly optimizes classification accuracy and self-explanation during training. OFA achieves this through a multi-objective framework, eliminating the need for separate explainability models while ensuring faithful and robust explanations. Extensive experiments on medical datasets confirm that OFA delivers competitive classification performance while providing high-quality, inherently interpretable explanations, making it a scalable and versatile solution for fully explainable classification.