CO2 Enhanced Oil Recovery (CO2-EOR) is a green and promising technology that not only improves crude oil recovery but also reduces carbon emissions. The micro-pore structure of porous reservoirs significantly influences the effectiveness of CO2-EOR, making research in this area crucial. Therefore, this study aims at the influence of pore heterogeneity on the immiscible CO2 displacement and simulates the microscale two-phase flow of CO2-oil displacement in porous media by a coupled approach of the N-S equations and phase-field method. This paper establishes three different heterogeneity models, by setting the degree of deviation from the average radius. The impact of pore heterogeneity on the shape and stability of the CO2-oil displacement front, residual oil types, and spatial distribution are investigated under different capillary numbers. The results indicate that the increased pore heterogeneity in porous media leads to a more unstable displacement front under conditions of neutral wettability (θ = π/2) and low capillary numbers (log10Ca = -6.253). The overall recovery increases with an increase in capillary numbers in the model with weak heterogeneity (σ = 0.004). However, the presence of dominant channels leads to a decrease in the oil recovery with the increasing capillary numbers in the models with stronger heterogeneity (σ = 0.008 and σ = 0.012). The findings of this research provide valuable insights into the mechanisms governing the impact of pore heterogeneity on the non-miscible displacement process of CO2-oil for CO2-EOR.