Effectively managing heat generation in high-power electronic devices and microelectromechanical systems is critical for their performance and reliability. This paper proposes and numerically optimizes an innovative microchannel heat sink (MCHS) design that incorporates 50 curved concave microchannels, each separated by a cylindrical microtube. The design aims to enhance heat transfer efficiency while addressing manufacturability. The MCHS, fabricated from copper, is cooled by a laminar, single-phase flow of a silver/water–ethylene glycol hybrid nanofluid (50:50), with nanoparticle volume fractions ( \(\phi\) ) ranging from 0 to 1% and Reynolds numbers (Re) between 100 and 400, under a base heat flux of 2300 W cm-2. A three-dimensional conjugate heat transfer model is solved using the finite volume method, with numerical results serving as the database for a subsequent optimization framework. The response surface method (RSM) coupled with a genetic algorithm (GA) is employed to optimize two key geometric parameters: the microtube diameter and its vertical position within the fin. The objective is to simultaneously minimize the maximum surface temperature (Tmax), improve temperature uniformity (Θ), and reduce the required pumping power (PP). Key quantitative results reveal that increasing the nanoparticle concentration to \(\phi\) = 1% enhances the overall heat transfer coefficient by approximately 10.93% at Re = 400. Optimization identifies an optimal configuration (microtube diameter = 60 μm, fin height = 40 μm) which reduces Tmax to 309.41 K and achieves Θ = 0.0167, representing a significant improvement over baseline designs. The predictive RSM models demonstrate high accuracy for thermal parameters, with maximum errors of 0.24% for Tmax and 10.18% for Θ, though the PP prediction error is higher at 19.38%. In conclusion, the integrated CFD-RSM-GA approach successfully delivers an optimized, high-performance MCHS design that balances enhanced thermal management with hydraulic efficiency, demonstrating strong potential for advanced electronic cooling applications.