Time series and sequential data are widespread in many real-world environments. However, implementing physical and adaptive dynamical systems remains a challenge. Direct Feedback Alignment (DFA) is a learning algorithm for neural networks that overcomes some of the limits of backpropagation and can be implemented in neuromorphic hardware (e.g., photonic accelerators). Until now, DFA has been investigated mainly for feedforward architectures. We adapt DFA for both “vanilla” and gated recurrent networks. Unlike backpropagation, the update rule of our DFA can be applied in parallel across time steps, thus removing the sequential propagation of errors. We benchmark DFA on 4 datasets for sequence classification tasks. Although backpropagation still achieves a better predictive accuracy, our DFA shows promising results, especially for environments and physical systems where backpropagation is unavailable.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Direct Feedback Alignment for Recurrent Neural Networks

  • Sara Folchini,
  • Andrea Cossu,
  • Andrea Ceni,
  • Vincenzo Lomonaco,
  • Davide Bacciu,
  • Claudio Gallicchio

摘要

Time series and sequential data are widespread in many real-world environments. However, implementing physical and adaptive dynamical systems remains a challenge. Direct Feedback Alignment (DFA) is a learning algorithm for neural networks that overcomes some of the limits of backpropagation and can be implemented in neuromorphic hardware (e.g., photonic accelerators). Until now, DFA has been investigated mainly for feedforward architectures. We adapt DFA for both “vanilla” and gated recurrent networks. Unlike backpropagation, the update rule of our DFA can be applied in parallel across time steps, thus removing the sequential propagation of errors. We benchmark DFA on 4 datasets for sequence classification tasks. Although backpropagation still achieves a better predictive accuracy, our DFA shows promising results, especially for environments and physical systems where backpropagation is unavailable.