Computed Tomography (CT) is a widely used imaging technique, but its reliance on ionizing radiation limits its applicability in repeated or sensitive medical/industrial scenarios. Ultrasound Computed Tomography (USCT) offers a safer alternative; however, the resulting reconstructions are often of low resolution and prone to artifacts, making anomaly detection challenging. In this work, we propose a two-stage deep learning pipeline (DLP) for anomaly classification and localization in simulated USCT images. Numerical experiments consider a viscoelastic material immersed in water with properties similar to human tissue. Training and validation datasets were primarily generated from synthetic handwritten images reconstructed via backprojection tomography (BP) and complemented with a small subset ( \(\approx 6\%\) ) of numerically simulated ultrasonic tomography reconstruction (FBP-USCT). In the first stage of DLP, a ResNet-50 model addresses a six-class classification task, distinguishing between images with and without anomalies, achieving \(90.16\%\) accuracy. In the second stage, a modified Faster R-CNN with VGG-16 as backbone is used for anomaly localization, reaching 85.65% detection accuracy. The proposed CNN achieved about \(87\%\) precision on a test set excluded from training and validation. These results is a proof-of-concept that the proposed CNN-based model can effectively perform the classification and detection tasks, thereby reducing the cost of numerical or experimental datasets of ultrasonic tomography.