Towards Complex Composite Object Detection Technique in Remote Sensing Imagery


Student thesis: Master's Thesis

View graph of relations


Related Research Unit(s)


Awarding Institution
Award date12 Jul 2023


Different from the common images which are captured at short range, remote sensing images are of large scale and macroscopic view. Achievements in remote sensing technology contribute to the bloom of high-quality remote sensing images (RSIs). Among these, High-resolution remote sensing images (HRSIs) provide strong support for many applications and object detection is one of the most interesting issues. Though various algorithms have been validated to be triumphant in common object detection tasks, such as face detection and pedestrian detection, the complex composite scenarios in RSIs (e.g., coal-fired power plant, airport) are still challenging due to multiple discrete parts with variable layouts leading to complex weak inter-relationship and blurred boundaries, instead of a clearly-defined single object in tasks like vehicle detection. Moreover, applying object detection algorithms on real large-scale remote sensing scenarios still suffers from complex surroundings and inconsistency of mapping. The lack of a generalized large-scale detection approach with high accuracy leads to low efficiency and inconvenience in real applications. To address the aforementioned issues, this thesis proposes remote sensing-targeted solutions for composite object detection according to the characteristics of RSIs and composite objects. The main work contents are as follows:

1) To address the composite object detection algorithm in RSIs, this thesis proposes an end-to-end framework, i.e., RElational Part Awareness Network (REPAN), to explore the semantic correlation between the globe and parts, and extract discriminative features among multiple parts. Specifically, this thesis first designs a part region proposal network (P-RPN) to locate discriminative yet subtle regions. With butterfly units (BFUs) embedded, feature-scale confusion problems caused by aliasing effects can be largely alleviated. Second, a feature relation transformer (FRT) plumbs the depths of the spatial relationships by part-and-global joint learning, exploring correlations between various parts to enhance significant part representation. Finally, a contextual detector classifies and detects parts and the whole composite object through multi-relation aware features, where part information guides to localize the whole object. This thesis collects three remote sensing object detection datasets to assess the proposed REPAN. Comprehensive experiments demonstrate the proposed REPAN reaches state-of-the-art performance with an accuracy of 89.49\% and a precision of 88.17\%, indicating the effectiveness and potential of the proposed REPAN.

2) To further generalize the proposed REPAN in real scenarios, this thesis proposes a new generalized Large-Scale composite object detection Workflow (LSW) based on REPAN. To tackle the scene complexity problems of large-scale remote sensing images involved complex surrounding textures, a tandem enhanced module (TEM) and a recursive connection are designed to embed into the backbone to enhance target feature representation. The tandem enhanced module contains both a channel enhanced sub-network (CEN) and a spatial enhanced sub-network (SEN) located beside the backbone in a tandem manner, to realize channel and spatial adaptive re-calibration. In the prediction process, to handle the large input pixels and the patch boundary effects, here an overlapping strategy and a feature-based clipping-out strategy are proposed. Further, LSW is applied in the Greater Bay Area in China with an area of 56,000 square kilometers and a long-time series of 20 years and processes a long-term coal-fired power plant detection from 2000 to 2020. Additional analysis based on the large-scale detection results is presented. By the scaling ability and transferring ability validation, the proposed LSW shows the advantages on accuracy and efficiency.