In recent years, there has been a substantial amount of projects focused on the difficult task of editing images in a way that is both accurate and manageable. In image editing, pixel-level interactive editing is one of the research directions. A typical paradigm for interactive editing is proposed by the point-based editing framework that is represented by DragGAN (Pan et al. 2023). In more recent times, the integration of drag operations and interactive editing, along with the generative abilities of diffusion models, has resulted in the development of even more superior solutions (Shi et al. 2024). The generation impact of this strategy, on the other hand, continues to be plagued by problems of low editing efficiency and low resilience of the procedure. The purpose of this work was to propose a new technique called DaptDiffusion, which aims to enhance the process of editing latent space based on diffusion model by introducing two extra strategies. 1) a Dense-UNet latent space feature network, and 2) a point update approach that was inspired by Adam. In addition to enhancing the editing efficiency and robustness of the approach, this method also improves the semantic and information transmission capabilities of the latent space feature network. We conduct experiments to illustrate the high quality and high usefulness of DaptDiffusion in many different circumstances. The graphic abstract can be shown in Fig. 1. Accessible source code can be found at the following location: https://github.com/Gdw040199/DaptDiffusion