Emotion recognition from multimodal data, such as audio and text, has garnered significant attention in recent years due to its potential applications in human-computer interaction, affective computing, and conversational AI. While traditional approaches rely on either unimodal or rudimentary fusion methods, they often fail to capture complex interdependencies across modalities, leading to suboptimal performance. This paper proposes a novel hybrid transformer architecture for robust multimodal emotion recognition that effectively integrates both audio and text modalities. The proposed model leverages an Audio Spectrogram Transformer (AST) for the audio modality and pre-trained Bidirectional Encoder Representations from Transformers (BERT) for the text modality. A unified multimodal transformer fusion mechanism is introduced to combine the representations learned from both modalities, facilitating the capture of cross-modal correlations. Despite the exclusion of the visual modality, which is commonly employed by state-of-the-art methods, experimental results on emotion recognition benchmarks demonstrate promising results with an F1 score of 0.67 on the IEMOCAP dataset that is composed of 12-hour-long audio recordings annotated with emotional labels. Our findings indicate that the proposed attention-based multimodal approach is both efficient and effective in emotion analysis, especially when applied to smaller datasets, marking a significant step forward in multimodal emotion recognition systems.