The effective management of large-scale datasets with diverse features is essential in data preprocessing for machine learning applications. Feature selection plays a critical role in ensuring the identification of the most relevant features, thereby improving model accuracy and insights. This paper investigates various feature selection techniques using a car insurance dataset, focusing on feature selection while considering an ordinal categorical feature as the target feature. Additionally, this study establishes a foundation for enhancing predictive models for customer classification by drawing insights from a real-world case study in the insurance industry. Four methods were compared: least absolute shrinkage and selection operator (LASSO), gradient boosting, stepwise regression, and univariate feature selection model. Results of validation indicate that LASSO outperforms other methods using the Bangdiwala statistic and its corresponding graph to evaluate the behavior of features in each model. This research underscores the significance of feature selection in refining insurance-related analytics and achieving a more comprehensive understanding of the field.