Physical literacy, encompassing movement skills, coordination, and confidence, is a critical yet underexplored domain in both sports and computer vision. In this paper, we introduce PLNet-12, a novel video dataset consisting of 12 fundamental movement skills designed to support the evaluation of physical literacy through multi-task vision-language models (VLMs). Unlike existing datasets, PLNet-12 includes fine-grained annotations for temporal action detection, repetition counting, displacement and distance measurement, duration estimation, and physical literacy scoring. We focus on zero-shot evaluation using large-scale VLMs without task-specific fine-tuning to investigate their generalization ability in real-world settings. To ensure annotation quality, we employ a semi-automated, rule-based pipeline verified by motion capture data and domain experts. We benchmark several state-of-the-art VLMs under different temporal and spatial settings and reveal challenges including fine-grained temporal understanding, action confusion, and temporal hallucination. Our findings highlight critical limitations of current VLMs in structured physical tasks and suggest directions for future multi-modal reasoning research. PLNet-12 offers a lightweight yet challenging platform for advancing temporal video understanding in human movement analysis.