Emotional information is present in every spoken audio event that individuals frequently hear. As a result, Speech Emotion Recognition (SER) has gained widespread recognition. Over the past ten years, this has grown into a significant research topic. Through the use of human voices or everyday conversation, SER can detect people’s emotional states. It is essential for developing Human-Computer Interaction (HCI) and signal processing systems. Emotions in humans also evolve with time. Therefore, to comprehend the dependencies in the speech sign over time, a strong model is required. In this work, Transformer-based method using the CNN model in parallel (TASER-Net) method is used for SER. With the use of a parallel CNN model, our novel approach to temporal emotion modelling for SER overcomes information loss from noise and bi-directional propagation while creating multi-scale contextual emotional representations across a range of time frames.