This study aimed to classify the emotions experienced using physiological signals and brain activity as participants viewed emotion-inducing videos. We conducted an ablation study of the physiological and brain activity data using Fast Fourier Transforms and concluded that brain activity yields the most discernible signals. We further applied ablation on the brain signals to identify the most relevant features of the dataset. The Voxel-Wise analysis showed that 63% of the brain signals are relevant for classification. After pinpointing the key datasets and features, we tested five different neural network models with hyperparameter tuning, as well as an algorithmic-based approach. The neural networks achieved 30% accuracy, while the algorithm achieved 76.6% accuracy in classifying emotions, demonstrating the ability to discern distinct emotions using physiological data. This shows that the simple classifier was able to outperform the more complicated neural network models.