Surface defects are a significant concern in the manufacturing industries, as they affect the functionality and performance of machined components. Traditional methods for multiple defects detection on milled surfaces rely on human inspection, which is subjective, time-consuming, and prone to errors. This research aims to address this challenge by developing multiple surface defects detection system based on machine vision techniques that utilize-focus variation-based optical profilometer to capture the images, and computer vision algorithms, to automatically detect and classify defects on milled surfaces as per ISO 8785 that includes scratches, pitting, deposits, erosion, and blowholes. This work involved machining straight slots using end milling on Inconel 617 alloy under varied machining conditions. You Only Look Once (YOLOv7), an object detection algorithm, was used for defect localization and detection. A total of 18,024 images, with 13,215 images for training (75%) and 4809 images for validation (25%), were used for developing the machine vision-based surface defect detection system. A total of 7020 images are used for validation. The precision and recall values of the model were observed to be around 43% and 45%, respectively, along with mean average precision coming around 40%.