<p>This work uses the response surface methodology (RSM) to determine the hyperparameters with which the XGBoost should work to improve its specificity in classifying patients with breast cancer. The algorithm was compared against two other machine learning algorithms, and its results were the worst compared to those of the other two algorithms. Therefore, the XGBoost hyperparameters were adjusted using the RSM to measure the effectiveness of the RSM in tuning the machine learning algorithms. A Box–Behnken experimental design was used to adjust the hyperparameters, and the response variable was specificity. The average specificity at the beginning was around <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11227_2025_7600_Article_IEq1.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="31" /> </InlineMediaObject> <EquationSource Format="TEX">$93\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>93</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> in the breast cancer problem. When applying the RSM to adjust the hyperparameters, there is an increase in specificity of more than <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11227_2025_7600_Article_IEq2.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="23" /> </InlineMediaObject> <EquationSource Format="TEX">$5\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>5</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation>, reaching more than <InlineEquation ID="IEq3"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11227_2025_7600_Article_IEq3.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="31" /> </InlineMediaObject> <EquationSource Format="TEX">$98\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>98</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> in this variable. Furthermore, compared to the other algorithms, there are also improvements. Better results in specificity are obtained with RSM compared to hyperparameter tuning using Metaheuristics, Grid Search, and Bayesian Optimization in this study.</p>

Response surface-driven hyperparameter optimization for XGBoost

期刊论文
发表日期：

2025年7月7日

查看全文

Jair Vasquez-Ramos,
María Guadalupe Ruiz-Sandoval,
Diego Oliva,
Oscar Ramos-Soto,
Jorge Ramos-Frutos,
Marwa Sharawi,
Marco Pérez-Cisneros

摘要

This work uses the response surface methodology (RSM) to determine the hyperparameters with which the XGBoost should work to improve its specificity in classifying patients with breast cancer. The algorithm was compared against two other machine learning algorithms, and its results were the worst compared to those of the other two algorithms. Therefore, the XGBoost hyperparameters were adjusted using the RSM to measure the effectiveness of the RSM in tuning the machine learning algorithms. A Box–Behnken experimental design was used to adjust the hyperparameters, and the response variable was specificity. The average specificity at the beginning was around $93\%$ $93 %$ in the breast cancer problem. When applying the RSM to adjust the hyperparameters, there is an increase in specificity of more than $5\%$ $5 %$ , reaching more than $98\%$ $98 %$ in this variable. Furthermore, compared to the other algorithms, there are also improvements. Better results in specificity are obtained with RSM compared to hyperparameter tuning using Metaheuristics, Grid Search, and Bayesian Optimization in this study.