Practical Gaze Tracking on Any Surface with Your Phone

This paper introduces ASGaze, a novel gaze tracking system using the RGB camera of smartphones. ASGaze improves the accuracy of existing methods and uniquely tracks gaze points on various surfaces, including phone screens, computer displays, and non-electronic surfaces like whiteboards or paper - a situation that is challenging for existing methods. To achieve this, we revisit the 3D geometric eye model, commonly used in high-end commercial trackers, and it has the potential to achieve our goals. To avoid the high cost of commercial solutions, we identify three fundamental issues when processing the eye model with an RGB camera, including how to accurately extract iris boundary that is the meta-information in our design, how to remove ambiguity from iris boundary to gaze point transformation, and how to map gaze points onto the target surface. Furthermore, as we consider deploying ASGaze in real-world applications, two additional challenges should be addressed: how to automatically and accurately annotate the training dataset to reduce manual labor and time costs, and how to accelerate the inference speed of ASGaze on mobile devices to improve user experience. We propose effective techniques to resolve these issues. Our prototype and experiments on three tracking surfaces demonstrate significant performance gains. The project site: https://asgaze.github.io. © 2024 IEEE