Speech Emotion Recognition (SER) is a critical component of human-computer interaction, with speech serving as a primary modality for understanding emotional states. In this study, we propose Whisper-KAN, a novel framework that combines the Whisper pretrained model for extracting log-Mel spectrogram-based embeddings with the Kolmogorov-Arnold Network (KAN) for interpretable and efficient emotion classification. Evaluated on the IEMOCAP dataset, our approach achieves a state-of-the-art results, surpassing conventional weight-based deep learning methods. The model demonstrates strong performance, particularly for nuanced emotions like anger and sad, as evidenced by Receiver Operating Characteristic (ROC) curve analysis.