Knee osteoarthritis (KOA) is a widespread degenerative joint disorder that significantly impacts patient mobility and quality of life. Early and accurate assessment of KOA severity is crucial for effective clinical decision-making and treatment planning. In this study, we propose the Swin Transformer Based Bidirectional Feature Pyramid Network (ST-BiFPN) that combines the multi-scale feature extraction capabilities of a Bidirectional Feature Pyramid Network (BiFPN) with the global contextual modeling strengths of the Swin Transformer. This integrated approach is designed to capture both the subtle local changes and broader structural variations present in knee X-ray images. Extensive evaluations on a comprehensive knee osteoarthritis dataset demonstrate that our method achieves superior performance with 71.14% accuracy, outperforming traditional convolutional neural network architectures by 7-9%. These promising results suggest that the ST-BiFPN offers a robust tool for the automated classification of KOA severity, paving the way for improved diagnostic support and personalized treatment strategies in clinical settings.