3-D Deformable Object Manipulation Using Deep Neural Networks

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

16 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number8769898
Pages (from-to)4255-4261
Journal / PublicationIEEE Robotics and Automation Letters
Volume4
Issue number4
Online published23 Jul 2019
Publication statusPublished - Oct 2019

Abstract

Due to its high dimensionality, deformable object manipulation is a challenging problem in robotics. In this letter, we present a deep neural network based controller to servo control the position and shape of deformable objects with unknown deformation properties. In particular, a multi-layer neural network is used to map between the robotic end-effector's movement and the object's deformation measurement using an online learning strategy, In addition, we introduce a novel feature to describe deformable objects' deformation used in visual servoing. This feature is directly extracted from the 3-D point cloud rather from the 2-D image as in previous work. In addition, we perform simultaneous tracking and reconstruction for the deformable object to resolve the partial observation problem during the deformable object manipulation. We validate the performance of our algorithm and controller on a set of deformable object manipulation tasks and demonstrate that our method can achieve effective and accurate servo control for general deformable objects with a wide variety of goal settings. Experiment videos are available at https://sites.google.com/view/mso-deep.

Research Area(s)

  • Deep Learning in Robotics and Automation, Dual Arm Manipulation, Model Learning for Control, RGB-D Perception, Visual Servoing

Citation Format(s)

3-D Deformable Object Manipulation Using Deep Neural Networks. / Hu, Zhe; Han, Tao; Sun, Peigen; Pan, Jia; Manocha, Dinesh.

In: IEEE Robotics and Automation Letters, Vol. 4, No. 4, 8769898, 10.2019, p. 4255-4261.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review