Compositional Action Recognition (CAR) aims to identify unseen verb–noun combinations during training, yet complex human-object interactions, diverse spatiotemporal dynamics, and distributional shifts pose significant challenges to compositional generalization. Although existing approaches have achieved notable progress, mainstream video baseline methods still suffer from performance bottlenecks, particularly in distinguishing fine-grained action categories, which reflects their limited capacity for temporal structure modeling. To address this issue, we propose a Temporally-Aware Multi-task Representation Learning (TAM) framework, by employing multi-task co-optimization, to encourage the model to actively construct temporal semantic structures. Specifically, three auxiliary tasks are introduced: (1) Forward-Reversed Discriminator (FRD), to enhance global temporal direction awareness; (2) Frame-Stage Classifier (FSC), to capture the fine-grained temporal structure within the action; (3) Temporally-Semantic Alignment (TSA), to further improve representations of temporal semantic and dynamic structural evolution. Extensive experiments on CAR benchmarks demonstrate the effectiveness of the proposed method.