Automatic Tracking of Surgical Instruments with a Continuum Laparoscope Using Data-Driven Control in Robotic Surgery

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations


Related Research Unit(s)


Original languageEnglish
Article number2200188
Journal / PublicationAdvanced Intelligent Systems
Issue number2
Online published23 Dec 2022
Publication statusPublished - Feb 2023



In existing surgery process, surgeons need to manually adjust the laparoscopes to provide a better field of view (FOV) during operation, which may distract surgeons and slow down the surgery process. Herein, a data-driven control method that uses a continuum laparoscope to adjust the FOV by tracking the surgical instruments is presented. A Koopman-based system identification method is first applied to linearize the nonlinear system. Shifted Chebyshev polynomials are used to construct observation functions that transfer low-dimension observations to high-dimension ones. The Koopman operator is approximated using a finite-dimensional estimation method. An optimal controller is further developed according to the trained linear dynamic model. Furthermore, a learning-based pose estimation framework is designed to detect keypoints on surgical instruments and provides visual feedback for the control system. Compared with other detection methods, the proposed scheme achieves a higher detection precision and provides more optional keypoints for tracking. Simulation and experiments validate the feasibility of the proposed control method. Experimental results show that the proposed method can automatically adjust the field of continuum laparoscope by tracking surgical instruments in real time and satisfy the clinical requirements.

Research Area(s)

  • continuum laparoscope, data-driven control, keypoint detection, robotic surgery, visual feedback, KOOPMAN OPERATOR, MODEL, SYSTEMS

Download Statistics

No data available