Gaze Tracking for Robotic Limb Control: Calibration, Estimation of POG and Tracking

Project: Research

View graph of relations


This project aims at developing the methodology and system in human gaze tracking forcontrolling a robotic upper limb for executing the intended actions by a handicappedperson. The model of human eyes will be studied, and the method will be developed todetect the motion intention for the upper limb from the human eyes. An efficient gazetracking method will be developed to achieve estimation of the points of gaze (POG) inreal time. A suitable calibration approach will be investigated for the gaze trackingsystem. Different from traditional gaze estimations, the gaze based tracking of a targetobject will incorporate 3D gaze estimation. The goal in the gaze estimation is to locatethe object of interest for further tracking and manipulation with a robotic arm.Furthermore, trajectory tracking of target object will be conducted in full 3D space. Thetracked object trajectories will be described with a new invariant based method to forrecognition and reproduction in further manipulation actions by the robotic limb.Through the study of this project, the theoretical framework will be established forhand-eye coordination for handicapped people with robotic upper limbs.The output of the project will be a novel solution to the control of a robotic limb by theintention of a physically handicapped person from his eyes. The results of the projectwill contribute to the development of technology and systems to provide better assistanceto disabled people. The study will have significant impact on the theoretical developmentand the practical applications in gaze and motion tracking. With the effectivemethodology and system to develop, some urgent needs can be met in integrating roboticlimb to a disabled person as an integral part of his body. This will be highly useful forimproving the quality of life for handicapped or disabled people via adopting themethodology in the related fields including robot-assisted rehabilitation and robot-enhancedtherapy. The project will also benefit the general works in human-robotinteractions with the new interface provided by the gaze tracking.


Project number9042345
Grant typeGRF
Effective start/end date1/01/17 → …

    Research areas

  • robotics , gaze estimation , calibration , tracking , human-robot interface