<p>Traditional metrics such as precision, recall, mean Average Precision (mAP), and F-score are widely used to evaluate object detection models. However, in some frame-based medical scenarios, these metrics often fail to capture the true effectiveness of models. For instance, in frame-based data, an object detection model may detect true positives in just a few frames, resulting in a perfect precision, but miss the same targets in other frames, leading to a high number of false negatives and, as a result, a very low recall. In practice, this model may still function effectively as a medical assistant and accurately identify critical features. Yet, the traditional metrics do not reflect this acceptable performance. This study aims to address this limitation by introducing a new evaluation metric tailored for frame-based medical object detection tasks. We propose the S-MEOD (Sequential Method of Evaluation for Object Detection), a novel metric that combines Sequence-aware Precision (SaP) and Sequence-oriented Detection (SoD) to provide a more comprehensive assessment of model performance. The metric was evaluated on frame-based sequences using object detection models, including YOLO-based architectures, with experiments on medical data. Experimental evaluations showed that S-MEOD provides a more accurate and intuitive reflection of model effectiveness in frame-based detection compared to traditional metrics. In our experimental evaluation on coronary angiography data, increasing the confidence threshold led to higher precision (up to 0.964) and mAP50 (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$\approx 0.49$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo>≈</mo> <mn>0.49</mn> </mrow> </math></EquationSource> </InlineEquation>), but caused recall to drop from (<InlineEquation ID="IEq2"> <EquationSource Format="TEX">$\approx 0.22$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo>≈</mo> <mn>0.22</mn> </mrow> </math></EquationSource> </InlineEquation>) to 0.028 and the F1-score from (<InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\approx 0.29$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mo>≈</mo> <mn>0.29</mn> </mrow> </math></EquationSource> </InlineEquation>) to 0.055; correspondingly, S-MEOD, where lower values indicate better performance, increased from 1.30 at low thresholds to 2.06 at high thresholds, indicating a substantial deterioration in temporal detection performance. Compared to traditional metrics, S-MEOD more accurately reflects clinically relevant detection behavior by distinguishing between sparse high-precision detections and genuine sequence-level detection failure. The S-MEOD offers an easy-to-interpret and reliable alternative to existing metrics for evaluating frame-based medical object detection models. Its adoption could improve the assessment of clinical applicability and redefine performance standards in medical imaging research.</p>

S-MEOD: A Novel Evaluation Metric for Frame-Based Medical Object Detection

期刊论文
发表日期：

2026年5月6日

查看全文

Isaac Honarmand Rad,
Seyedreza Taghizadeh

摘要

Traditional metrics such as precision, recall, mean Average Precision (mAP), and F-score are widely used to evaluate object detection models. However, in some frame-based medical scenarios, these metrics often fail to capture the true effectiveness of models. For instance, in frame-based data, an object detection model may detect true positives in just a few frames, resulting in a perfect precision, but miss the same targets in other frames, leading to a high number of false negatives and, as a result, a very low recall. In practice, this model may still function effectively as a medical assistant and accurately identify critical features. Yet, the traditional metrics do not reflect this acceptable performance. This study aims to address this limitation by introducing a new evaluation metric tailored for frame-based medical object detection tasks. We propose the S-MEOD (Sequential Method of Evaluation for Object Detection), a novel metric that combines Sequence-aware Precision (SaP) and Sequence-oriented Detection (SoD) to provide a more comprehensive assessment of model performance. The metric was evaluated on frame-based sequences using object detection models, including YOLO-based architectures, with experiments on medical data. Experimental evaluations showed that S-MEOD provides a more accurate and intuitive reflection of model effectiveness in frame-based detection compared to traditional metrics. In our experimental evaluation on coronary angiography data, increasing the confidence threshold led to higher precision (up to 0.964) and mAP50 ( $\approx 0.49$ $\approx 0.49$ ), but caused recall to drop from ( $\approx 0.22$ $\approx 0.22$ ) to 0.028 and the F1-score from ( $\approx 0.29$ $\approx 0.29$ ) to 0.055; correspondingly, S-MEOD, where lower values indicate better performance, increased from 1.30 at low thresholds to 2.06 at high thresholds, indicating a substantial deterioration in temporal detection performance. Compared to traditional metrics, S-MEOD more accurately reflects clinically relevant detection behavior by distinguishing between sparse high-precision detections and genuine sequence-level detection failure. The S-MEOD offers an easy-to-interpret and reliable alternative to existing metrics for evaluating frame-based medical object detection models. Its adoption could improve the assessment of clinical applicability and redefine performance standards in medical imaging research.