CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts

Fuyuki Tokuda; Akira Seino; Akinari Kobayashi; Kazuhiro Kosuge

doi:10.1109/ICRA48891.2023.10160635

CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts

Fuyuki Tokuda
, Akira Seino
, Akinari Kobayashi
, Kazuhiro Kosuge

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

6 Citations (Scopus)

Abstract

This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator's motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme. © 2023 IEEE.

Original language	English
Title of host publication	2023 IEEE International Conference on Robotics and Automation (ICRA)
Publisher	IEEE
Pages	748-754
Number of pages	7
ISBN (Electronic)	979-8-3503-2365-8
ISBN (Print)	979-8-3503-2366-5
DOIs	https://doi.org/10.1109/ICRA48891.2023.10160635
Publication status	Published - 2023
Externally published	Yes
Event	40th IEEE International Conference on Robotics and Automation (ICRA 2023) - ExCeL London, London, United Kingdom Duration: 29 May 2023 → 2 Jun 2023 https://www.icra2023.org/

Publication series

Name	Proceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)	1050-4729

Conference

Conference	40th IEEE International Conference on Robotics and Automation (ICRA 2023)
Abbreviated title	ICRA2023
Place	United Kingdom
City	London
Period	29/05/23 → 2/06/23
Internet address	https://www.icra2023.org/

Funding

This work was supported in part by the Innovation and Technology Commission of the HKSAR Government under the InnoHK initiative. The research work described in this paper was in part conducted in the JC STEM Lab of Robotics for Soft Materials funded by The Hong Kong Jockey Club Charities Trust.

Access Output

10.1109/ICRA48891.2023.10160635

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{7833b0f3e7e341288fda8aae1cb608a9,

title = "CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts",

abstract = "This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator's motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme. {\textcopyright} 2023 IEEE.",

author = "Fuyuki Tokuda and Akira Seino and Akinari Kobayashi and Kazuhiro Kosuge",

year = "2023",

doi = "10.1109/ICRA48891.2023.10160635",

language = "English",

isbn = "979-8-3503-2366-5",

series = "Proceedings - IEEE International Conference on Robotics and Automation",

publisher = "IEEE",

pages = "748--754",

booktitle = "2023 IEEE International Conference on Robotics and Automation (ICRA)",

address = "United States",

note = "40th IEEE International Conference on Robotics and Automation (ICRA 2023) , ICRA2023 ; Conference date: 29-05-2023 Through 02-06-2023",

url = "https://www.icra2023.org/",

}

Tokuda, F, Seino, A, Kobayashi, A & Kosuge, K 2023, CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts. in 2023 IEEE International Conference on Robotics and Automation (ICRA). Proceedings - IEEE International Conference on Robotics and Automation, IEEE, pp. 748-754, 40th IEEE International Conference on Robotics and Automation (ICRA 2023) , London, United Kingdom, 29/05/23. https://doi.org/10.1109/ICRA48891.2023.10160635

CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts. / Tokuda, Fuyuki; Seino, Akira; Kobayashi, Akinari et al.
2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023. p. 748-754 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts

AU - Tokuda, Fuyuki

AU - Seino, Akira

AU - Kobayashi, Akinari

AU - Kosuge, Kazuhiro

PY - 2023

Y1 - 2023

N2 - This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator's motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme. © 2023 IEEE.

AB - This paper proposes CNN-based visual servoing for simultaneous positioning and flattening of a soft fabric part placed on a table by a dual manipulator system. We propose a network for multimodal data processing of grayscale images captured by a camera and force/torque applied to force sensors. The training dataset is collected by moving the real manipulators, which enables the network to map the captured images and force/torque to the manipulator's motion in Cartesian space. We apply structured lighting to emphasize the features of the surface of the fabric part since the surface shape of the non-textured fabric part is difficult to recognize by a single grayscale image. Through experiments, we show that the fabric part with unseen wrinkles can be positioned and flattened by the proposed visual servoing scheme. © 2023 IEEE.

UR - https://www.scopus.com/pages/publications/85168708150

UR - https://www.scopus.com/pages/publications/85168708150#tab=impact

U2 - 10.1109/ICRA48891.2023.10160635

DO - 10.1109/ICRA48891.2023.10160635

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 979-8-3503-2366-5

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 748

EP - 754

BT - 2023 IEEE International Conference on Robotics and Automation (ICRA)

PB - IEEE

T2 - 40th IEEE International Conference on Robotics and Automation (ICRA 2023)

Y2 - 29 May 2023 through 2 June 2023

ER -

CNN-based Visual Servoing for Simultaneous Positioning and Flattening of Soft Fabric Parts

Abstract

Publication series

Conference

Funding

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this