HVAC systems account for significant energy consumption in buildings, while conventional control strategies often lack consideration for occupant-related information, potentially leading to both energy waste and thermal discomfort. Although recent human-in-the-loop control advances aim to balance energy efficiency with occupant comfort, existing approaches predominantly rely on fixed occupancy schedules and remain limited to small-scale environments with few occupants. Large-scale spaces with multiple devices and numerous occupants pose significant optimisation challenges, while simultaneously presenting considerable research value and energy-saving potential. This study proposes a Dueling Double Deep Q-Network (D3QN) framework for joint thermal comfort and energy optimisation, implemented in a large open office environment with 10 Fan Coil Units (FCUs) serving 18 workstations. The D3QN agent dynamically adjusts temperature setpoints for each FCU by processing real-time environmental data and occupancy status collected through a distributed IoT sensor network. Experimental results demonstrate the system’s capability to optimize HVAC operations in high-dimensional scenarios while adapting to dynamic spatial-occupancy patterns.