While there is a strong focus on the energetic renovations of building to reduce carbon emissions, still a large number of buildings do not meet the set targets. On top of that, the perception exists that the replacement of gas boilers by heat pumps is only possible when an advanced energetic renovation has taken place. This study evaluates the viability of heat pump installations in partially renovated buildings through simulations conducted in Dymola (Modelica). The study compares the energetic performance of a residential terraced building equipped with a gas boiler, an air-to-water heat pump, or an air-to-air heat pump. Two different stages of renovation are considered in the comparison (intermediate and advanced), as well as two levels of thermal comfort (low and high). The results show that even in intermediately renovated buildings, there are opportunities to replace gas boilers by heat pumps. Air-to-water heat pumps are able to meet the thermal comfort criteria while reducing primary energy use and operational cost, if the electricity-to-gas price ratio is favorable. Compared to gas boilers at least 3.2 times less energy is used. After advanced renovations, air-to-air heat pumps emerge as the most efficient alternative, using up to 34.8% less electricity than air-to-water heat pumps. However, in intermediately renovated buildings, thermal comfort is not always met, stressing the importance of assessing the compatibility of indoor units with the room’s heat load.