Hand Disambiguation in the Egocentric Perspective


Student thesis: Doctoral Thesis

View graph of relations

Related Research Unit(s)


Awarding Institution
Award date29 Sept 2020


With the recent development of wearable cameras, the interest for research on the egocentric perspective is increasing. When addressing the egocentric perspective we encounter finding the hands as a challenging problem to solve, hence the first step focuses on disambiguate the hands from the background and then disambiguate between the hands (left/right), as the hands are the most consistent objects in the egocentric perspective and they can represent more information about people and their activities.

However, recent progress in egocentric hand disambiguation and even hand detection, especially using deep learning, has been limited by the lack of a large enough dataset, with suitable variations in subjects, activities, and scenes. We introduce a dataset that simulates daily activities, with variable illumination and people from different cultures and ethnicity to address daily life conditions.

To approach the hand disambiguation problem we propose using egocentric hand prop­erties that add information for hand disambiguation. We propose using hand uniqueness to use the information that only one of each hand appears in the image, mirrored shape to use the information that the left hand's shape is similar to the mirrored right hand's shape, and the hand context, that uses the information around the hands (arms and other objects). We also use previous hand properties introduced in previous works, like hand location that uses the region where the hand appears in the image, hand size that uses the width and height the hand presents, and the hand image probability, that uses the probability of each hand appearing in the image.

Finally, we proposed applying these egocentric hand properties into a Neural Network architecture. We propose different architectures combining the proposed properties and we provide baseline results with current object/hand detection approaches.

    Research areas

  • Hand detection, Attention, Object Proposal, Object Selection, Egocentric perspective, Hand disambiguation, Neural networks (Computer science), Convolutional Neural Networks