Doctors often disagree when classifying shoulder fractures—a fact supported by studies on observer agreement varying in their design, case selection, and the observers under study. Adding advanced imaging modalities, experienced observers, or reducing the number of categories has not improved the agreement to an acceptable level. Should we be concerned about these findings, and what is the potential impact on clinical practice and patient outcomes? Reliable classification systems can help integrate high-quality evidence in clinical decision-making, as results from randomized trials rely on classification systems. This chapter surveys the various types of observer studies, their findings, and limitations. I argue for the need to reduce the number of future observer studies to avoid research waste.