<p>Multiple sensing and processing capabilities enable an organism to obtain comprehensive information and make real-time decisions in unstructured environments. Inspired by the biological counterpart, developing neuromorphic hardware with multimodal fusion is supposed to realize high-precision processing for complex information. In this work, we developed a visual-auditory-fused reservoir computing system by combining a CH<sub>3</sub>NH<sub>3</sub>PbI<sub>3</sub> nanowire/pentacene optoelectronic memristor and a sound sensor. Under the stimulation of optical/electrical signals, the CH<sub>3</sub>NH<sub>3</sub>PbI<sub>3</sub> nanowire-based device exhibited volatility under optical or electrical stimulation, providing an essential foundation for visual-auditory-fused reservoir computing. The motion detection function was demonstrated using synchronous visual and auditory sensory information. Compared with single sensory input, the detection accuracy increased from 51.2% to 95.5% by combining the visual and auditory modalities. The results presented in this work provide a promising strategy to develop efficient multimodal neuromorphic hardware for human-robot interactions.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimode-fused reservoir computing based on CH3NH3PbI3 nanowire optoelectronic memristor for spatiotemporal information processing

  • Yue Wang,
  • Xuanyu Shan,
  • Qiang Wang,
  • Zewei Wang,
  • Siyu Liu,
  • Wei Liu,
  • Jiahui Zheng,
  • Xiaoning Zhao,
  • Zhongqiang Wang,
  • Ya Lin,
  • Ye Tao,
  • Haiyang Xu,
  • Yichun Liu

摘要

Multiple sensing and processing capabilities enable an organism to obtain comprehensive information and make real-time decisions in unstructured environments. Inspired by the biological counterpart, developing neuromorphic hardware with multimodal fusion is supposed to realize high-precision processing for complex information. In this work, we developed a visual-auditory-fused reservoir computing system by combining a CH3NH3PbI3 nanowire/pentacene optoelectronic memristor and a sound sensor. Under the stimulation of optical/electrical signals, the CH3NH3PbI3 nanowire-based device exhibited volatility under optical or electrical stimulation, providing an essential foundation for visual-auditory-fused reservoir computing. The motion detection function was demonstrated using synchronous visual and auditory sensory information. Compared with single sensory input, the detection accuracy increased from 51.2% to 95.5% by combining the visual and auditory modalities. The results presented in this work provide a promising strategy to develop efficient multimodal neuromorphic hardware for human-robot interactions.