Reconstructing Children’s Faces from Cropped Lip and Nose Images: A Large Mask Inpainting Approach for Privacy-Conscious Cleft Data
摘要
Cleft lip and Palate (CLP) is a congenital anomaly that leads to deformities in the orofacial regions at birth. While Cleft deformities can be fully corrected through primary Cleft repair surgery followed by secondary interventions, accurately representing the outcomes of primary Cleft repair is crucial for planning subsequent treatments. However, current surgical assessment methods are subjective and often result in discrepancies among clinicians. Although deep learning solutions have been explored as alternatives for providing objective measurements, the limited availability of Cleft data—primarily due to privacy concerns—raises concerns about their accuracy in real-world applications. Transfer learning with pretrained facial deep learning models, which are typically trained on full-face images, cannot be directly applied to Cleft conditions because clinical Cleft datasets generally consist of cropped facial images focusing on Cleft-prone areas, lacking full facial representations. To address this challenge, we propose a pipeline that generates synthetic full-face images of children from these cropped Cleft prone facial regions. The proposed inpainting model preserves Cleft-prone regions while generating semantically appealing whole face regions with continuity. These synthetic images serve as a valuable resource for analyzing anomalies using pretrained facial deep learning models trained on whole faces, which have demonstrated success in identifying fine and subtle facial details. Additionally, our approach enhances patient privacy and mitigates risks associated with model inversion attacks.