Panoramic dental radiographs (OPG) are the only imaging modality that captures the entire dentition in a single exposure. To support dentists with diagnosing caries it is essential to find indications for cavities on those images. While recent deep learning methods show strong results on in-distribution test sets, the generalization on out-of-distribution datasets is mostly untested. In this study, we suggest a two-stage deep learning pipeline for caries detection on single-tooth images extracted from OPGs: (1) image-level classification using a DINO-based transformer backbone and (2) instance-level segmentation using Mask-R-CNN. We perform experiments on data from the University Medical Center Schleswig-Holstein (UKSH). To study generalization, we test models on an out-of-distribution set from the Federal University of Bahia (UFBA) and also evaluate a mixed-domain setting including both UKSH and UFBA data. Further we investigate the influence of strong augmentation techniques. Results show that classification performance is high on in-distribution data but significantly drops when applied to out-of-distribution samples. Segmentation performance is moderate across all settings, with limited robustness under domain shift. These findings suggest that in-distribution results overestimate real-world performance and underscore the importance of evaluating domain shifts in dental AI pipelines.