Instance space analysis (ISA) is a methodology developed to visualize the space of all possible instances for testing an algorithm, showing where algorithm performance is known based on empirical evidence, and how algorithm performance can be predicted across the entire instance space using machine learning. As a primary goal, ISA supports visual insights into the strengths and weaknesses of algorithms under various test instance conditions, and is therefore a critical tool to establish algorithmic trust. As a secondary goal, ISA enables the diversity and potential biases of benchmark suites of test instances to be scrutinized, and the identification of gaps in the instance space where new instances would be valuable to generate or source. The instance space can be used to guide the generation of new test instances with desirable properties using evolutionary algorithms to enable rich and comprehensive suites of test instances, ensuring algorithm performance is understood under the widest range of conditions. The ability of an evolutionary algorithm to generate test instances along the mathematically defined boundary of the instance space also provides valuable insights into the tightness of existing upper and lower bounds on test instance features and, in the style of experimental mathematics, can generate new conjectures in fields such as graph theory. Based on a combination of linear algebra, optimization, statistics, and machine learning methods, ISA has been applied to a wide variety of problems in optimization, machine learning, time series forecasting, and other fields. This chapter provides an overview of the ISA methodology and how it provides the above-mentioned insights via several case studies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Instance Space Analysis for Visualization of Algorithmic Trust

  • Kate Smith-Miles,
  • Jeffrey Christiansen

摘要

Instance space analysis (ISA) is a methodology developed to visualize the space of all possible instances for testing an algorithm, showing where algorithm performance is known based on empirical evidence, and how algorithm performance can be predicted across the entire instance space using machine learning. As a primary goal, ISA supports visual insights into the strengths and weaknesses of algorithms under various test instance conditions, and is therefore a critical tool to establish algorithmic trust. As a secondary goal, ISA enables the diversity and potential biases of benchmark suites of test instances to be scrutinized, and the identification of gaps in the instance space where new instances would be valuable to generate or source. The instance space can be used to guide the generation of new test instances with desirable properties using evolutionary algorithms to enable rich and comprehensive suites of test instances, ensuring algorithm performance is understood under the widest range of conditions. The ability of an evolutionary algorithm to generate test instances along the mathematically defined boundary of the instance space also provides valuable insights into the tightness of existing upper and lower bounds on test instance features and, in the style of experimental mathematics, can generate new conjectures in fields such as graph theory. Based on a combination of linear algebra, optimization, statistics, and machine learning methods, ISA has been applied to a wide variety of problems in optimization, machine learning, time series forecasting, and other fields. This chapter provides an overview of the ISA methodology and how it provides the above-mentioned insights via several case studies.