Gaze Tracking for Robotic Limb Control: Calibration, Estimation of POG and Tracking

Project: Research

View graph of relations


This project aims at developing the methodology and system in human gaze tracking for controlling a robotic upper limb for executing the intended actions by a handicapped person. The model of human eyes will be studied, and the method will be developed to detect the motion intention for the upper limb from the human eyes. An efficient gaze tracking method will be developed to achieve estimation of the points of gaze (POG) in real time. A suitable calibration approach will be investigated for the gaze tracking system. Different from traditional gaze estimations, the gaze based tracking of a target object will incorporate 3D gaze estimation. The goal in the gaze estimation is to locate the object of interest for further tracking and manipulation with a robotic arm. Furthermore, trajectory tracking of target object will be conducted in full 3D space. The tracked object trajectories will be described with a new invariant based method to for recognition and reproduction in further manipulation actions by the robotic limb. Through the study of this project, the theoretical framework will be established for hand-eye coordination for handicapped people with robotic upper limbs.The output of the project will be a novel solution to the control of a robotic limb by the intention of a physically handicapped person from his eyes. The results of the project will contribute to the development of technology and systems to provide better assistance to disabled people. The study will have significant impact on the theoretical development and the practical applications in gaze and motion tracking. With the effective methodology and system to develop, some urgent needs can be met in integrating robotic limb to a disabled person as an integral part of his body. This will be highly useful for improving the quality of life for handicapped or disabled people via adopting the methodology in the related fields including robot-assisted rehabilitation and robot-enhanced therapy. The project will also benefit the general works in human-robot interactions with the new interface provided by the gaze tracking.


Project number9042345
Effective start/end date1/01/17 → …

    Research areas

  • robotics , gaze estimation , calibration , tracking , human-robot interface