Controllable instance synthesis with hierarchical regularization for semi-supervised pedestrian detection

Generative Adversarial Network (GAN)-based pedestrian image synthesis opens up the possibility of using synthesized data with sufficient diversity to cover unseen variations, which facilitates semi-supervised pedestrian detection. To improve synthesized image quality and diversity in the semi-supervised setting, we propose a Hierarchical Regularized GAN (HiR-GAN), which allows a controllable instance generation process. In the proposed model, a pedestrian instance is synthesized and stitched on top of a background image. By imposing shape-based regularization on intermediate results and complete pedestrian images, a generator learns to disentangle pedestrian shape and appearance without any supervision. As a result, we achieve effective control of background, shape, and appearance via explicit latent codes. Further, the generator competes with two discriminators, which focus more on pedestrian shape and image quality, respectively. By jointly training a background-pedestrian classifier, the generator is guided to capture more precise semantics of real pedestrian data, while at the same time the classifier is able to produce more accurate annotations of unlabeled data. Experiment results on multiple benchmarks demonstrate the superiority of HiR-GAN over previous state-of-the-arts in semi-supervised pedestrian image synthesis and the effectiveness of high-fidelity synthesized data in reducing the dependence on labeled data in the pedestrian detection task. © 2024 Elsevier B.V.