In complex component assembly tasks, hand motion dynamics critically determine assembly precision. Existing approaches predominantly fail to capture the synergy between dynamic contours and motion trajectories, limiting their performance in complex scenarios. To address these issues, this paper proposes a Dynamic Hand Behavior Augmented Network (DHA-Net) for assembly procedure recognition, enhancing dynamic behavior representation and procedural semantic correlation. First, an adaptive hand-focus module dynamically adjusts the perceptual weights of hand contours and motion trajectories based on assembly phase characteristics, improving task adaptability. Second, a three-dimensional hand state tensor unifies the encoding of keypoint trajectories, directional cosines, and procedural labels, enabling explicit modeling of multi-scale dynamic semantics through temporal convolution and directional offset normalization. Finally, a novel two-stream network combining TSM and hand-centric sequence modeling effectively recognizes first-person complex assembly actions. Experimental results demonstrate that DHA-Net significantly outperforms existing methods in complex assembly scenarios, achieving notable accuracy improvements. By explicitly modeling dynamic hand behaviors and procedural semantics, this study provides an efficient, adaptive framework for intelligent assembly systems.