In the era of digital communication, the identification of regional dialects has become increasingly important, especially in the context of Arabic language processing. This study focuses specifically on Libyan Arabic Dialect (LAD), aiming to develop effective methods for distinguishing it from other Arabic dialects and Modern Standard Arabic (MSA) in scenarios where labeled data resources are limited. We evaluate five machine learning algorithms—Logistic Regression, K-Nearest Neighbors (KNN), Random Forest, AdaBoost, and Gradient Boosting—in conjunction with advanced Natural Language Processing (NLP) techniques. Our analysis explores various feature representations, including uni-grams, bi-grams, and tri-grams, combined with six term weighting schemes: one unsupervised scheme (TF-IDF) and five supervised schemes (TF-Chi2, TF-IG, TF-GR, TF-OR, and TF-RF). The primary objective is to identify the most effective combination of features and models for LAD classification under conditions of limited labelled data resources. Our findings indicate that Logistic Regression achieves the highest stability score (0.88) and composite score (0.74), along with an accuracy of 88% and an F1 score of 70%. While Random Forest demonstrates superior accuracy (89%) and F1 score (73%), it incurs higher computational costs. Notably, the integration of uni-grams with supervised term weighting schemes (TF-OR and TF-RF) yields the most effective configurations, achieving accuracy levels above 85% and F1 scores close to 75%. This study introduces a Composite Score metric, which integrates traditional evaluation metrics (accuracy, precision, recall, F1 score) with execution time to provide a holistic assessment of model performance. Logistic Regression and KNN emerge as optimal choices for real-time applications due to their balance of performance and efficiency (0.01 s execution time). These results contribute to the field of dialect identification by highlighting the importance of selecting appropriate machine learning models and term weighting schemes, with potential applications in language processing, sentiment analysis, and regional language resource development, particularly in scenarios with limited labeled data resources.