<p>The presented study focuses on the secondary data analysis of non-Newtonian carboxymethyl cellulose (CMC)-based Fe₂O₃ and Al₂O₃ nanofluids, where a data-driven approach was employed to evaluate and predict the thermal performance enhancement of a shell-and-helical coil heat exchanger. The effect of various operating conditions on the Overall Heat Transfer Coefficient (OHTC) was systematically evaluated. The test results showed significant improvements in the OHTC for both CMC/Fe₂O₃ and CMC/Al₂O₃ nanofluids with OHTC improvements of 24.3 and 26%, respectively, at 1 wt% under maximum flow conditions. The improvement comes from the increased thermal conductivity of the nanofluids, increased convective mixing due to turbulence and increased shear-thinning behaviour of the base fluid. To accurately predict the thermal performance of CMC/Fe₂O₃ and CMC/Al₂O₃, machine learning (ML) models (Ridge, Lasso, XGBoost and LightGBM) were developed and compared. Among these models, the XGBoost model performed the best (R² = 0.989) and is very predictive. The relationship of each of the features was further analysed with the support of Pearson correlation and Principal Component Analysis. Using Shapley Additive Explanations (SHAP) based interpretability, it was determined that the flow rate had the largest impact on the Nusselt Number (Nu), with flow rates having contributions between − 80 to + 100 units across 1–5 LPM. The stirrer speed (1200–1500 RPM) contributed significantly to augmenting the Nu due to the presence of turbulence/convection interaction, and by raising the temperature of the nanofluid (40–60&#xa0;°C), there was an increase of up to 40 units of SHAP. An experimental-AI-explainable hybrid framework such as this will achieve high prediction accuracy as well as physical insight. The proposed hybrid experimental–AI–explainable framework provides both high prediction accuracy and physical insight, offering a robust approach for optimizing nanofluid-based heat exchangers in advanced thermal systems.</p>