Humans possess an extraordinary capacity for facial recognition, enabling them to identify blurred or distant faces even after long intervals. Traditionally, in both public and private organizations, tasks like monitoring and facial recognition have been performed manually. However, long working hours and distractions can significantly impair the efficiency and focus required for consistent image monitoring. Consequently, there is an increasing demand for an efficient and automated face detection and recognition system across various sectors, including security, forensics, and defense. With advancements in deep learning, convolutional neural networks (CNNs) have dramatically improved the face recognition process in surveillance systems. In this paper, the convolutional base of VGGNet-16 is leveraged and enhanced with three customized layers, to process images from the FaceSurv dataset for facial recognition. This approach addresses challenges posed by limited training data and considers crucial factors such as subject distance and lighting variations. Furthermore, anomaly detection plays a vital role in identifying outliers, particularly in scenarios involving unknown or unauthorized subjects. To tackle this, an Isolation Forest technique is employed as a tree ensemble method, to accurately detect anomalies. The proposed methodology, tested on 120 known subjects, achieves a Recognition Accuracy of 97.5% for known subjects and effectively detects unknown subjects with zero misclassification.