Bridges are essential for enabling movement during environmental disasters and serve as crucial links for rescue and aid delivery. Effective bridge inspection and maintenance are more critical than ever due to increasing severity and frequency of environmental disasters. Although current state-of-the-art deep learning models have achieved good performance many challenges still exist, such as their performance on challenging datasets and their opaque-box nature makes it difficult to understand their decision-making process and identify potential biases. This research work proposes a novel architecture that incorporates innovative parallel twin attention module, synchronous amplification module, aggregated multi-feature attention module and squeeze and excitation blocks, that helps to focus on specific regions of the image plane automatically resulting in improved structural component recognition accuracy. Its parallelism helps to capture long-range dependencies enabling the model to use contextual information encompassing spatial and channel information when segmenting bridge components. Experimental results and ablation studies show that our proposed architecture outperforms the current state-of-the-art methodologies in the challenging bridge component classification dataset. We also examine our models through XAI methods to provide insights into its decision-making process and making it more trustable by highlighting the importance of different features for various similar recognition/segmentation tasks.