From a neural and cognitive standpoint, learning is known to rely on top-down information that conveys expectations about sensory events. Deep neural networks, however, have focused primarily on a bottom-up approach, where supervised learning adjusts weights based on the discrepancy between the network output and a desired target. To account for the role of both top-down and bottom-up learning, we developed a novel learning rule termed Top-Down Backpropagation (TDBP) where supervised learning is combined with a top-down component that aims to reduce the difference between the network output and expectations that are gradually adjusted during training. Compared with a standard approach, TDBP attained lower training error, better testing accuracy, and improved robustness to adversarial data. We discuss the implications of these findings for the design of artificial neural networks that account for the influence of both bottom-up sensory information and top-down expectations during learning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Top-Down Backpropagation in Deep Feedforward Neural Networks

  • Artem Pilzak,
  • Jean-Philippe Thivierge

摘要

From a neural and cognitive standpoint, learning is known to rely on top-down information that conveys expectations about sensory events. Deep neural networks, however, have focused primarily on a bottom-up approach, where supervised learning adjusts weights based on the discrepancy between the network output and a desired target. To account for the role of both top-down and bottom-up learning, we developed a novel learning rule termed Top-Down Backpropagation (TDBP) where supervised learning is combined with a top-down component that aims to reduce the difference between the network output and expectations that are gradually adjusted during training. Compared with a standard approach, TDBP attained lower training error, better testing accuracy, and improved robustness to adversarial data. We discuss the implications of these findings for the design of artificial neural networks that account for the influence of both bottom-up sensory information and top-down expectations during learning.