Multi-timescale Processing with Heterogeneous Assembly Echo State Networks
摘要
An echo state network (ESN) is a reservoir computing framework that consists of an input layer, reservoir, and readout layer. Recently, Iinuma et al. proposed the assembly echo state network (AESN), which mitigates the increased computational load accompanying an increase in the number of neurons under high-dimensional input, thereby outperforming conventional ESNs. The development of new ESN architectures faces challenges associated with high dimensionality and tasks involving multiple timescales, necessitating multi-timescale capabilities. However, in conventional AESNs optimized for high-dimensional inputs, the parameters of the time-scale dynamics are set to be common among the individual ESNs. To enhance performance when learning tasks with multiple timescales, we introduce a novel AESN, referred to as a Heterogeneous AESN (HetAESN), comprising multiple sub-reservoirs driven by different timescale dynamics. Our experimental results show that by adjusting the timescales of the sub-reservoirs, HetAESN outperforms conventional ESNs and AESNs in two coupled van der Pol (tc-VdP) tasks, selected as benchmark tasks for time-series prediction involving different time scales. Consequently, the proposed method can contribute to the application of AESNs in general learning tasks involving multiple timescales.