With the intensification of global climate change and energy crises, building energy conservation and carbon reduction have become a crucial pathway to achieving sustainable development. This paper investigates the impact of eight influencing factors-building orientation, exterior wall structure, transparent envelope, window-to-wall ratio, roof structure, roof insulation, shading forms, and architectural forms-on the energy consumption of high-rise residential buildings in five cities across Hainan province: east, west, south, north, and central. The research employs methods of on-site surveys, dynamic energy consumption simulations, and orthogonal experiments. Results indicate that roof structure and window-to-wall ratio are the most significant factors affecting energy consumption, followed by building orientation and architectural form. Significant energy savings can be achieved through the optimization of passive design factors. The energy-saving rates for Haikou, Dongfang, Sanya, Qionghai, and Qiongzhong cities are 68.84%, 64.15%, 54.28%, 56.35%, and 47.96%, respectively. Moreover, a building energy consumption prediction model is constructed, which effectively predicts energy consumption and facilitates the rapid generation of decision-making schemes. The findings provide a scientific basis for the optimization of passive energy-saving measures in high-rise residential buildings in Hainan, contributing to the sustainable development of the construction industry and the achievement of energy conservation and emission reduction goals.