In this paper, we propose an incremental update strategy for continuous action iterated hierarchical dilemma (CAIHD). This strategy overcomes the fixed player number limitation in traditional dynamic model of CAIHD. Firstly, we introduce an incremental update strategy that enables the model to reflect dynamic changes in player number. Secondly, we note that the standard incremental update strategy redundantly refreshes existing player states and suffers from low computational efficiency. To resolve this issue we develop an improved update strategy that prevents unnecessary changes to original player states during asymmetric learning. Finally, we apply a design-based Lyapunov method to prove convergence of evolutionary dynamics for CAIHD under the proposed update strategy. The effectiveness of the proposed strategy is further confirmed through simulation experiments.