The carbon market mechanism (CETM) is a vital approach to promoting global low-carbon economic development. To investigate its emission reduction effectiveness, this study takes the CETM pilot policies as a quasi-natural experiment. Using panel data of 27 sub-industrial sectors from 30 provinces in China during 2007–2019, we constructed two models based on industrial structure and carbon regulation intensity. From the perspective of regional heterogeneity, the inhibitory effect of CETMs on carbon emissions in regions with a high industrial proportion is weaker than that in regions with a low industrial proportion. Additionally, the emission reduction effect of CETMs is more significant in regions with strong environmental control compared to those with weak environmental control. We aims to provide policy references for improving the setting of a differentiated CETM system and exploring carbon emission reduction paths for different industrial sectors in various regions.