Histopathology refers to the study of a disease at a cellular level which stands as a golden method of predicting breast cancer. In this paper, a comparative study of the performance of machine learning models trained using optimized feature sets is done. The experiments are conducted using two datasets. The first is the Wisconsin Breast Cancer dataset, which contains 30 extracted features of cell nuclei. The second is the MITOS-ATYPIA 14 dataset, consisting of histopathology images, from which hand-crafted features have been extracted. Population based metaheuristic optimization algorithms are used to optimize and choose the key features from the available feature set to increase the efficacy of the model. Support vector machines, logistic regression model and other classification models are tested using this optimized feature set. To evaluate the impact of feature optimization, accuracy, precision, recall, and F1 score are assessed using both the full feature set and the optimized subset from two datasets. The results demonstrate how model performance varies with different feature sets, underscoring the significance of optimization techniques in enhancing machine learning-based breast cancer diagnosis in medical imaging.