The advent of the Exascale era is transforming supercomputing, integrating new workflows that encompass simulations, data science, and artificial intelligence. This evolution necessitates a critical re-evaluation of how the supercomputing community assesses performance. To address this, a dedicated workshop was convened with the central aim of exploring and debating the future of performance benchmarks. The workshop featured presentations and interactive panel discussions, drawing expertise from prominent benchmarking communities. Key figures and stakeholders from industry, academia, and government participated, fostering a comprehensive dialogue on the necessity and desirability of developing a benchmark suite that accurately reflects and accommodates these emerging applications. This paper summarizes the insights and discussions from this pivotal workshop, offering a state-of-the-art overview of the ongoing efforts to evolve supercomputing benchmarks.