Echo state network (ESN) is a family of reservoir computing (RC) realizing energy-efficient learning. In an ESN, a reservoir maps time-series input signals nonlinearly to high-dimensional space, which is utilized for performing various tasks. Many studies have analyzed reservoirs on stability, nonlinearity, and memory capacity. Among them, the memory capacity has been analyzed and evaluated as the integral of a determination coefficient, which is obtained as a function of time delay. In this paper, by focusing on the determination-coefficient function (DCF), we elucidate the relationship between memory length and respective hyperparameters, namely, the number of neurons, activation function, network connectivity, and the spectral radius in a reservoir. We conduct two experiments assuming tasks of multi-frequency signal separation and multi-variance noise grouping. The experiments show that the DCF profile is of significant importance in determined by the degree of concentration of the reservoir state to the spherical surface in the high dimension information space depending on the spectral radius. The results also demonstrate that ESNs organize proper clusters, having a high representation ability even with a spectral radius smaller or larger than unity. These findings help us to determine the spectral radius in reservoirs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Utilizing Small and Large Spectral Radii for Appropriate Reservoir Computing Design

  • Bungo Konishi,
  • Akira Hirose,
  • Ryo Natsuaki

摘要

Echo state network (ESN) is a family of reservoir computing (RC) realizing energy-efficient learning. In an ESN, a reservoir maps time-series input signals nonlinearly to high-dimensional space, which is utilized for performing various tasks. Many studies have analyzed reservoirs on stability, nonlinearity, and memory capacity. Among them, the memory capacity has been analyzed and evaluated as the integral of a determination coefficient, which is obtained as a function of time delay. In this paper, by focusing on the determination-coefficient function (DCF), we elucidate the relationship between memory length and respective hyperparameters, namely, the number of neurons, activation function, network connectivity, and the spectral radius in a reservoir. We conduct two experiments assuming tasks of multi-frequency signal separation and multi-variance noise grouping. The experiments show that the DCF profile is of significant importance in determined by the degree of concentration of the reservoir state to the spherical surface in the high dimension information space depending on the spectral radius. The results also demonstrate that ESNs organize proper clusters, having a high representation ability even with a spectral radius smaller or larger than unity. These findings help us to determine the spectral radius in reservoirs.