Speaker recognition involves the process of recognizing an individual by analysing their voice. It has received significant interest for its potential uses in various fields, such as security systems, voice assistants, and personalized services. To get accurate speaker recognition, the proposed Speaker Recognition system (SR) uses both the deep learning model LSTM and the conventional machine learning algorithms. The system extracts time, frequency, and cepstral features from audio signals and trains different classification models, including Logistic Regression, Random Forest, Naïve Bayes, Support Vector Machines, En- semble and LSTM models. Performance evaluation across two speech datasets (Librispeech and Audio MNIST) reveals that the LSTM model consistently outperforms other classifiers in the Librispeech dataset with an accuracy score of 98.19%. Meanwhile, in the audio MNIST dataset, traditional machine learning classifiers perform better than the LSTM model with the accuracy score above 95%. The analysis reveals that the LSTM model excels in large datasets with complex pattern, while traditional machine learning models exhibit a higher performance on structured or simpler audio data. Mel Frequency Cepstral Coefficients (MFCC) features demonstrate high performance across various classifiers and datasets, highlighting their effectiveness in capturing essential speaker-discriminative information.