TIRgel sensor has pioneered a new visuo-tactile sensing mechanism that integrates visual and tactile modalities through the remote/near vision conversion of the focus-adjustable camera. To further expand its tactile function, this paper proposes a 3D reconstruction method, MLP-Depth, based on the photometric stereo. First, we press the sensor with a calibration ball with a known size. Then, the corresponding depth value of each pixel is obtained by calibrating the contact area and resolving the sphere geometry. Finally, a direct mapping between RGB and depth is constructed based on MLP. Experiments show that the MLP-Depth has high reconstruction precision, especially for unknown objects, with high reconstruction robustness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MLP-Depth: An Improved Visuo-Tactile 3D Reconstruction Method Applied to TIRgel Sensor

  • Shixin Zhang,
  • Yuhao Sun,
  • Funchun Sun,
  • Huaping Liu,
  • Yiyong Yang,
  • Bin Fang

摘要

TIRgel sensor has pioneered a new visuo-tactile sensing mechanism that integrates visual and tactile modalities through the remote/near vision conversion of the focus-adjustable camera. To further expand its tactile function, this paper proposes a 3D reconstruction method, MLP-Depth, based on the photometric stereo. First, we press the sensor with a calibration ball with a known size. Then, the corresponding depth value of each pixel is obtained by calibrating the contact area and resolving the sphere geometry. Finally, a direct mapping between RGB and depth is constructed based on MLP. Experiments show that the MLP-Depth has high reconstruction precision, especially for unknown objects, with high reconstruction robustness.