When evaluating lesion localization performance in medical imaging, methods like the Free-response Receiver Operating Characteristic (FROC) often fall short for segmentation-based AI predictions where each pixel has a continuous score. While adaptations exist, they involve implicit design choices with significant implications for model development and clinical application. This paper examines one set of these design choices, the transformations required to discretize the AI prediction into a set of detections, and then determining whether those detections are successful or not. This is typically done by applying a threshold to a respective measure. With a set of experiments on both real-world and simulated datasets, we examine the interaction between these thresholds and the resulting FROC score. Notably, we observe that this relationship is highly dependent on particular aspects of the problem case, and provide recommendations on how to address this.