Visual Inertial Segmentation and Ego-motion Estimation using Photometric Feedback

基於光度反饋的圖像分層及自我運動估計

Student thesis: Doctoral Thesis

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Awarding Institution
Supervisors/Advisors
Award date17 Jan 2024

Abstract

Micro aerial vehicles (MAVs), or drones, have gained tremendous popularity in recent decades in a broad spectrum of civil and military applications. However, smaller, human-friendly MAVs still struggle to carry multiple sensors such as LiDAR and cameras for autonomous navigation because of limited payload and power budget. Thus, reactive navigation, an efficient vision-based strategy has been proposed for ego-motion estimation for small drones.

This thesis focuses on the development of reactive navigation for computationally and payload limited MAVs. Our system is designed such that the quadrotor is equipped with only a monocular camera and inertial sensors. Our solution allows the robot to efficiently estimate its ego-motion in indoor environments.

To achieve efficient and robust ego-motion estimation, we propose a single step featureless method to estimate the state of the robot. In this work, we rely on the presence of planar surfaces such as walls and grounds. This permits us to assume that the camera can only see one plane in a static environment. The novelty primarily lies in the implementation of the gradient-based featureless approach and the direct use of photometric feedback in the state and output vectors. Under the single-plane assumption, the proposed framework allows the entire estimation process to be accomplished in a single iterative step without the need for feature detection and tracking or pre-computation of optic flow as required in previous works. We conducted extensive flight experiments with different flight patterns to evaluate the performance of the proposed observer. The results reveal that the root-mean-square error in the altitude estimates is approximately 10% for flights at 40 cm to 130 cm above the ground, comparable to those from feature-based estimators.

To extend our work to accommodate scenarios where multiple planes might be present, we relax the single plane assumption. The research focuses on the plane segmentation. Unlike the prevailing semantic image segmentation methods which are based on neural networks that require prior training, our method focuses on the visual observable changes in consecutive image frames. This substantially reduces the computational load and broadens the potential field. We conducted several fight experiments in real corridors to maintain a flight altitude of 0.48 m, while staying 0.6 m from the wall. Even with corrupted initial estimates, the quadrotor can gradually converge to the correct flight path.

Overall, the proposed system enables drones with low computation ability to robustly estimate its ego-motion. Scientific merits of the work include the developments of (i) a gradient-based featureless approach. This eliminates the feature detection and tracking process, allowing the entire estimation to be achieved in a single step; and (ii) a novel approach for image segmentation using only directly observable data.