A Self-Supervised Transformer With Feature Fusion for SAR Image Semantic Segmentation in Marine Aquaculture Monitoring

The rapid development of the marine aquaculture industry has brought about a series of environmental problems that need to be monitored and planned. There is abundant marine aquaculture data obtained through synthetic aperture radar (SAR) remote sensing over a long period. With a large amount of unlabeled data, self-supervised learning can describe the feature representation of targets. However, when self-supervised learning meets big data, it often leads to semantic information loss, such as interclass misjudgment and intraclass discontinuity. To address this issue, this article proposes a self-supervised transformer with feature fusion (STFF) for the semantic segmentation of SAR images in marine aquaculture monitoring. STFF consists mainly of a self-attention encoding module with a hybrid loss function and a semantic segmentation decoding module with feature fusion. For encoding, the transformer is pretrained via self-supervised learning based on a hybrid loss function to enrich local, global, and edge information for dealing with semantic information loss and data imbalance in whole-scene SAR images. For decoding, the features extracted from transformer blocks are fused to enhance semantic characteristics, improve the intraclass continuity of segmentation, and reduce the occurrence of interclass misjudgment. The superiority of the proposed method to state-of-the-art algorithms is demonstrated via experimentation on GaoFen-3 and Radarsat-2 SAR datasets. The code has been available at https://github.com/fjc1575/Marine-Aquaculture/tree/main/STFF-code for the sake of reproducibility. © 2023 IEEE.