The emergence of autonomous, Agentic AI systems has rendered traditional, single-turn evaluation benchmarks inadequate for assessing security. This chapter addresses this critical evaluation gap by providing a comprehensive guide to modern methodologies for testing Agentic AI. It begins by defining the core problem, the Capability-Security Paradox, where increased functionality creates an exponential rise in security complexity. The chapter then conducts a deep dive into two foundational frameworks: GAIA, for measuring general-purpose capabilities, and Stanford AIR, for structured risk assessment. To provide a holistic view, it surveys a broader ecosystem of specialized benchmarks targeting threats like agent hijacking and real-world vulnerability exploitation. A key contribution is the introduction of systematic risk quantification through the OWASP Agentic AI Top 10 and its corresponding AIVSS-Agentic scoring system. The central thesis concludes that effective evaluation must shift from auditing final outputs to securing the entire agentic process, advocating for an integrated strategy that combines capability testing, adversarial validation, and standardized risk scoring to foster the development of safe and trustworthy autonomous systems.