Towards Higher Quality in Expert Reports in Child Protection: Why Emphasizing Limitations and Alternative Explanations Matters
摘要
Admitting one’s limitations and providing alternative explanations for one’s own conclusions (i.e., justifying) in order to avoid prejudice and confirmation bias is important in most fields, especially when experts evaluate child protective cases. However, we lack knowledge about the relationship between these expressions and the quality of experts’ reports in such cases. In this study, we examined whether experts who are better at justifying their conclusions differ from those who are worse at justifying their conclusions in terms of the quality of their reports. We collected and coded, after reliability testing (Krippendorff’s α = .89), 306 expert reports regarding quality and justification-related variables. Spearman’s rho indicated that the overall quality ratings were positively related to the degree of justification (p < 0.001). Evaluation quality and methodological discussion were also associated with justification (p < 0.001) and differed among experts with low vs. middle vs. high levels of justification. Furthermore, rating scores of experts’ conduct and behavior increased with higher levels of justification (p < 0.001). Experts who discussed doubt and limitations in their work and who presented alternative hypotheses achieved higher scores on several of our quality measures. The higher scores on the judgement scale were further associated with a higher score on the experts’ code of conduct scale.