As power density becomes the main constraint of multicore systems, managing power consumption using DVFS while providing the desired performance becomes increasingly critical. Reinforcement learning (RL) performs significantly better than conventional methods in performance-power optimization under different hardware configurations and varying software applications. RL agents learn through trial-and-error by receiving rewards which is defined by an objective function (e.g. instructions-per-second (IPS)) within specified constraints (e.g. power budget). System and application requirements lead to changing objectives and constraints which in turn result in different reward functions. The RL agents adapt to these changing objectives and constraints (and hence reward functions). Equivalent-policy invariant comparison (EPIC) is a popular technique to evaluate different reward functions. EPIC provides a numerical score which quantifies the difference in two reward functions. In this work, we use this EPIC distance (score) to transfer knowledge and improve learning for changing reward functions. Experimental results using a DVFS enabled RISCV based system-on-chip implemented on an FPGA shows 16.2% lower power budget overshoots compared to a tabular Q-learning agent with direct transfer.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

EPIC-Q: Equivalent-Policy Invariant Comparison Enhanced Transfer Q-learning for Run-Time SoC Performance-Power Optimization

  • Anmol Surhonne,
  • Haitham S. Fawzi,
  • Florian Maurer,
  • Oliver Lenke,
  • Michael Meidinger,
  • Thomas Wild,
  • Andreas Herkersdorf

摘要

As power density becomes the main constraint of multicore systems, managing power consumption using DVFS while providing the desired performance becomes increasingly critical. Reinforcement learning (RL) performs significantly better than conventional methods in performance-power optimization under different hardware configurations and varying software applications. RL agents learn through trial-and-error by receiving rewards which is defined by an objective function (e.g. instructions-per-second (IPS)) within specified constraints (e.g. power budget). System and application requirements lead to changing objectives and constraints which in turn result in different reward functions. The RL agents adapt to these changing objectives and constraints (and hence reward functions). Equivalent-policy invariant comparison (EPIC) is a popular technique to evaluate different reward functions. EPIC provides a numerical score which quantifies the difference in two reward functions. In this work, we use this EPIC distance (score) to transfer knowledge and improve learning for changing reward functions. Experimental results using a DVFS enabled RISCV based system-on-chip implemented on an FPGA shows 16.2% lower power budget overshoots compared to a tabular Q-learning agent with direct transfer.