Replicability and reproducibility are critical challenges in systems research, particularly in evaluating systems that experience performance variability due to hardware and complex self-regulating behaviours. This paper investigates performance evaluation practices for stream processing systems, focusing on the impact of backpressure. Backpressure occurs when data is received faster than it can be processed, leading to cascading delays and potential data loss. Through empirical analysis, we demonstrate where popular closed-loop benchmark designs used in benchmarks such as NEXMark and YCSB under backpressure conditions may fail to meet target arrival rates, leading to unreliable benchmarking results. Our study provides recommendations for metrics to better understand system behavior, and proposes best practices for reliable performance evaluation in the presence of backpressure.