Purpose <p>The purpose of this study was to reveal the conformity and test–retest reliability of the Kellgren–Lawrence (KL) grade evaluation ChatGPT-5.0 and orthopedic surgeons.</p> Methods <p>100 anterior–posterior knee radiographs from 100 subjects with knee pain were uploaded to ChatGPT-5.0. When the radiographs were uploaded, ChatGPT was asked with the same prompt for all images, “What is the Kellgren–Lawrence grade of this knee?”. The worst KL grade evaluated by ChatGPT was regarded as the answer. ChatGPT was asked the same question twice on 7&#xa0;days after the first trial. All radiographs were evaluated by two orthopedic surgeons with more than 20&#xa0;years of experience prior to the ChatGPT evaluation. The diagnoses by the surgeons and the ChatGPT answers were compared. The intra-(ChatGPT–ChatGPT) and inter-(ChatGPT–surgeons) rater reliabilities were calculated with the linear weighted kappa coefficient.</p> Results <p>ChatGPT could evaluate the KL grade based on joint space narrowing, bony spur formation, bone sclerosis, and joint deformity with the just uploaded knee radiograph. For the intra-rater reliability, Kappa coefficient of ChatGPT and orthopedic surgeon was 0.551, and 0.840, respectively. For the inter-rater reliability, Kappa coefficient was 0.644 to 0.730.</p> Conclusion <p>ChatGPT was able to evaluate the KL grade of the knee simply by uploading the radiograph image. The ChatGPT answer was significantly correlated with the orthopedic surgeons’ diagnosis. However, the intra-rater reliability of the ChatGPT’s answer was at a moderate level. The inter-rater reliability between ChatGPT and orthopedic surgeons’ diagnosis showed a substantial level.</p>