Customer complaint is a form of feedback for dissatisfaction of customer with a specific service or product. It has great impact on long term success and reputation of companies. In this paper, the proposed model assists the financial institutions in managing customer complaints and feedback to enhance customer satisfaction. It consists of three phases: pre-processing phase, transformer encoder phase and classification phase. In the pre-processing phase, the input text of user complaint is converted into numerical vectors after removing stop words. The feature extraction phase transforms complaints into encoded vector that fed as input to classification phase. It consists of embedding layer, positional encoding layer, multi-head attention, residual connection, normalization layer and feedforward layers. The classification phase consists of one average pooling layer and a pipeline of three fully connected layers. The experimental results prove that the proposed model achieved acceptable average classification performance, with 92% for precision, recall and F1-score and 94%, for accuracy across five categories comparing with state-of-art. The proposed model outperformed most of machine learning classifiers that extract feature with traditional methods as Term Frequency—Inverse Document Frequency (TF-IDF) and Countvetor.