Open QBench: A benchmarking framework for evaluating quantum computing platforms
摘要
The diverse landscape of quantum computing modalities and software frameworks poses significant challenges for evaluating performance across a range of computational tasks and applications. Benchmarking procedures for quantum computers are often intricate and difficult to reproduce for end-users, quantum algorithm developers, and quantum resource providers. This challenge is compounded by the emergence of analog, non-universal approaches to quantum information processing, including quantum annealers, boson samplers, and quantum simulators. Recent advances in quantum computing technology underscore the increasing need for well-defined, comprehensive, and standardized methods for performance benchmarking. This paper introduces an open and modular software framework to enhance the reproducibility and execution of quantum benchmarking experiments. The core utility of the framework lies in enabling seamless, standardized execution of benchmarks across diverse quantum computing modalities—including gate-based, photonic, and annealing QPUs—at various levels of the quantum-classical stack. As a practical demonstration of current capabilities, we present a novel methodology to deploy an application-driven benchmarking suite. To the best of our knowledge, no other benchmarking framework attempts to cover all three described modalities across different benchmarking levels. While some approaches enable such comparisons at a certain level, these analyses often do not provide a complete picture and must be complemented with additional and hardware specific metrics. Our framework, based on a curated set of representative problems for different user communities, is executed at both the circuit level and the hybrid classical-quantum level to provide a comprehensive assessment of quantum system performance. Finally, we propose an approach for analyzing performance results using multiple-criteria decision analysis (MCDA), which allows us to incorporate different performance metrics into a unified decision-making process that supports more transparent and interpretable benchmarking results.