<p>Periodic fin structures are often employed to enhance heat transfer in compact cooling solutions and heat exchangers. Adjoint-based optimization methods are able to further increase the heat transfer by optimizing the fin geometry. However, obtaining optimal geometries remains challenging in general because of the high computational cost of full array simulations. In this paper, a unit-cell optimization approach is presented that starts from recently developed macroscale models for isothermal solid structures. The models exploit the periodicity of the problem to reduce the computational cost of evaluating the array heat transfer to that of a single periodic unit-cell. By combining these models with a geometrically constrained free-shape optimization approach, optimal fin geometries are obtained for the periodic fin array that maintain a minimal fin distance. Moreover, using an augmented Lagrangian approach, also the average pressure gradient and barycenter of the fin can be fixed. On a fictitious use-case, heat transfer increases up to 104% are obtained. When also flow rate is constrained in addition to maintain a high effectiveness, only up to 8% heat transfer increase is observed. Finally, the errors of the unit-cell optimization approach are investigated, indicating that with a good choice of cost functional formulation, errors of the approach as low as 1–2% can be obtained for the periodically developed part of the array. Finally, the entrance effect to the heat transfer is found to be non-negligible with a contribution of 10–15% for the considered fin array. This advocates for further research to extend the unit-cell models toward improved modeling of entrance effects.</p>