<p>Carbon fiber reinforced polymer (CFRP) has been widely used in civil aviation owing to its excellent mechanical properties and lightweight characteristics. However, drilling operations during assembly may introduce defects into CFRP laminates, thereby compromising the structural reliability of aircraft. To the best of our knowledge, no image-based methods have yet been proposed for identifying surface defects around CFRP-drilled holes. Nevertheless, such recognition is of great importance, since the industry employs different criteria for assessing the severity of defects to ensure drilled-hole quality. To address this issue, this study first summarizes the formation mechanisms and visual characteristics of three types of drill-induced CFRP defects. An economical image acquisition strategy is then presented to capture high-quality images of defects around holes. Subsequently, a pure image-processing-based method for defect recognition is proposed. Specifically, binarization, contour detection, and denoising are applied to extract the hole center and radius; clustering and secondary denoising are employed to roughly segment the damaged region; and finally, three proposed indicators—the normalized minimum radial distance, circular connectivity score, and defect occupancy ratio—are used in an exclusion manner to identify burr, burn, and tear defects. Drilling experiments on CFRP laminates and validation with the collected images demonstrate that the proposed method can effectively recognize defects around drilled holes.</p>