Application-oriented benchmarking of quantum computing requires the evaluation of multiple, often conflicting, Key Performance Indicators (KPIs). To assess the overall quality of a single solution or to compare multiple solutions, these KPIs must be synthesized into a comprehensive metric. Multi-Criteria Decision Aiding (MCDA) provides a suitable methodological framework for this purpose. MCDA involves two main steps to produce an overall score from a vector of KPIs: first, normalizing the KPIs, and second, aggregating the normalized scores. It is important to note that normalization is not uniquely defined and may correspond to different measurement scales. We therefore examine the property of invariance, where switching from one admissible scale to another should not alter the comparative ranking of options. To facilitate aggregation, it is often assumed that some form of commensurability (or comparability) exists across the normalized KPI scores. However, this is a strong assumption that may not always hold in practice. As such, we investigate various aggregation functions that can accommodate different invariance assumptions, providing a more nuanced and robust approach to decision-making in quantum benchmarking.