As Japan faces large-scale infrastructure reconstruction due to aging systems and population decline, understanding the environmental implications of water infrastructure decisions becomes critical to avoid carbon lock-in effects. This study presents the first integrated assessment of CO2 emissions across the entire water supply chain from intake to tap, examining how distribution methods and urban structure interact to determine total emissions. Using a four-step methodology, we analyzed 11 water utilities serving Tokyo and Kanagawa Prefecture, calculating emissions from both utility-managed processes and building-level water supply systems. Results reveal a paradox: while Tokyo achieves lower emission intensity (229.165 g-CO2/m3) than Kanagawa (234.545 g-CO2/m3) from intake to distribution through systematic gravity flow implementation, this advantage reverses when including water supply to buildings, with Tokyo’s per-building emissions (124.448 kg-CO2/year) exceeding Kanagawa’s (111.763 kg-CO2/year). In Kawasaki City, despite achieving the lowest emission intensity (121.545 g-CO2/m3), districts like Musashi-Kosugi with vertical development generate extreme emissions due to energy-intensive pumping requirements. These findings indicate that current approaches optimizing distribution systems independently of urban planning fail to minimize total emissions. Effective decarbonization of water infrastructure requires integrated planning that considers building heights and future urban development patterns alongside technological improvements, ensuring that investments in efficient distribution systems align with the vertical reality of modern cities.