Cost-sensitive machine learning techniques are frequently used by data mining researchers to improve their models. In banking credit scoring in particular, the financial cost of misclassification leads us to pay particular attention to cost-sensitive modelling. Knowing that feature selection is a crucial preprocessing step in machine learning because it greatly affects a model’s capacity for prediction and generalization, it becomes important to propose financial cost-sensitive feature selection methods. In this paper, we propose Cost Sensitive Association Rules Feature Selection more Larger (ARFSL-CS), which is a two-phase feature selection algorithm based on Sequential Forward Selection (SFS). In the first phase, SFS selects the attributes according to a confidence threshold of the association rules sensitive to the financial cost whose consequent is the class of the loan and in which they are in the antecedent. In the second phase, SFS selects from the remaining features those which minimize the financial cost of misclassification of the model when applied to a dataset described only by each of the features. Experiments carried out on three datasets, using a cost-sensitive Gradient Boosting credit scoring model, show that ARFSL-CS selects the attribute subsets that most minimize the overall cost of financial misclassification for the banker with a gain of 8.7%, 21%, and 69.9% compared to the prediction model without feature selection in the three considered datasets.