A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection

Sewer pipes are essential infrastructure for discharging wastewater. Regular pipe inspection is necessary to prevent malfunction of sewer systems, for which closed-circuit television (CCTV) crawlers are commonly used to capture images of the pipe interior. As manual assessment of pipe condition is labor-intensive and time-consuming, automated defect detection using computer vision and deep learning has been increasingly studied in recent years. However, deep learning approaches require large amount of annotated data for model training. Data collection in underground sewer pipes is expensive and difficult since they are inaccessible without the use of an inspection robot. Meanwhile, ground-truth annotation needs to be accurate and consistent, requiring massive time and expertise. This paper proposes a framework for synthetic data generation and augmentation to address the data shortage problem for sewer pipe defect detection. First, synthetic images of sewer pipes are generated by 3D modeling and simulation in virtual environment. The quality of the generated images is then enhanced using style transfer with reference to real inspection images. In addition, a contrastive learning module is developed to further improve the deep learning process for defect detection. Experiment results show that the average precision (AP) of the defect detection model is improved by 2.7% and 4.8% respectively after adding style-transferred synthetic images and applying the contrastive module. When both methods are applied, the AP of the model is boosted by 7.7%, from 22.22% to 23.92%, indicating the effectiveness of our proposed approaches. This study is expected to alleviate the burden on data collection and annotation for applying deep learning models in defect detection. © 2022