<p>Orthopaedic training necessitates an objective and scalable evaluation of technical skills. The prevailing methodology depends on labor-intensive expert assessments and small, heterogeneous datasets. This study introduces a deep learning pipeline that transforms procedural video, optionally integrated with kinematic data, into calibrated ordinal and continuous estimates aligned with Objective Structured Assessment of Technical Skills and Global Rating Scale scores. The pipeline utilizes large-scale self-supervised and weakly supervised pretraining, incorporating temporal contrastive learning, masked video modeling, and phase and tool cues, alongside an orthopaedic-specific curriculum and distribution alignment across simulators, cadavers, benchtop models, and operating rooms. Skill prediction employs an uncertainty-aware ordinal head and heteroscedastic regression with post hoc calibration, facilitating selective prediction and principled deferral in high-stakes environments. Phase and tool-conditioned spatiotemporal attributions offer interpretable, formative feedback for learners. The model is paired with an evaluation framework that addresses validity, reliability, transportability, educational utility, and fairness through institution and surgeon-aware splits, transport tests, and pretrain on X test on Y scenarios. Metrics encompass rank correlation, mean absolute error, ordinal accuracy, calibration error, and proxy measures such as phase F1 score, tool average precision, and temporal F1 at k. Compared with classical feature-based methods, 3D convolutional networks, Transformers, and surgical self-supervised learning baselines, our domain-adapted pretraining enhances calibration, label efficiency, and maintains competitive accuracy while abstention policies mitigate errors at moderate coverage.</p>