A deep learning network based end-to-end image composition

Currently, high-quality image composition largely depends on multiple user interactions and complex manual operations. In particular, the process of composition object extraction and region determination has become a burden that cannot be underestimated, restricting wider applications. Aiming at this problem, we propose an end-to-end image composition method that combines powerful deep-learning-based application modules such as image retrieval and instance segmentation to realize efficient non-interactive image composition. Specifically, the retrieval module, which is based on the attention mechanism, can determine semantically similar material images. Moreover, the content of interest (COI) extraction and optimization procedure is able to select the most proper instance among the material images. Finally, we propose the double-sieving strategy, which locates the best composition position in the target image. Using these effective modules, we carried out niche targeting experiments using an image database with high plausibility. The realistic experimental results illustrate that our method can achieve effective and reasonable end-to-end image composition. © 2021 Elsevier B.V.