Self-supervised learning is one of the recent innovations in machine learning used in computer vision fields. Few attempts have so far used this approach to design and learn a skeleton for a mechanical system with complex kinematics, such as a robot manipulator. This work demonstrates novel applications of this method for pose estimation in this domain. The proposed model consists of two stages: a self-supervised stage designed to produce semantic segmentation exemplarily for an excavator, and a traditional image processing stage to convert the segmentation to a skeleton. An investigation using two types of contrastive self-supervised learning approaches shows that the DenseCL model gives a more accurate result in comparison to the SimCLR model. Accordingly, in the first stage, DenseCL is used to learn the basic structure of the object, and then K-Net associated with DeepLabV3 is used for the segmentation task. In the second stage, traditional image processing tools are applied to classify the segments and find the nearest intersection points to the boundary regions to create the skeleton. For a model based on ResNet-50, this model surpasses the previous baseline model in this field by 23.6% AP.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Self-supervised Segmentation to Pose Estimation Model for Mechanical Systems with Complex Kinematics

  • Ala’a Alshubbak,
  • Cyrille Ewoudi Ewoudi,
  • Daniel Görges

摘要

Self-supervised learning is one of the recent innovations in machine learning used in computer vision fields. Few attempts have so far used this approach to design and learn a skeleton for a mechanical system with complex kinematics, such as a robot manipulator. This work demonstrates novel applications of this method for pose estimation in this domain. The proposed model consists of two stages: a self-supervised stage designed to produce semantic segmentation exemplarily for an excavator, and a traditional image processing stage to convert the segmentation to a skeleton. An investigation using two types of contrastive self-supervised learning approaches shows that the DenseCL model gives a more accurate result in comparison to the SimCLR model. Accordingly, in the first stage, DenseCL is used to learn the basic structure of the object, and then K-Net associated with DeepLabV3 is used for the segmentation task. In the second stage, traditional image processing tools are applied to classify the segments and find the nearest intersection points to the boundary regions to create the skeleton. For a model based on ResNet-50, this model surpasses the previous baseline model in this field by 23.6% AP.