HumanDiff: Leveraging body-part expert-assisted diffusion transformers for human image generation

Synthesizing realistic human images remains a longstanding challenge in computer vision due to the highly articulated structure and diverse appearances of the human body. This task demands generating high-quality images while preserving semantic consistency across body parts and capturing fine-grained visual details. However, existing approaches often struggle to achieve precise alignment between semantic guidance and visual output, particularly when integrating multi-source semantic information or reconstructing local structures faithfully. To address these issues, we introduce HumanDiff, a novel diffusion transformer framework for structured human image generation. HumanDiff leverages multi-modal data through a unified feature extraction pipeline to deliver precise, semantically-grounded generation. A specialized expert module focuses on modeling individual body parts, while a soft routing mechanism dynamically fuses information across experts, ensuring smooth transitions and coherent integration between adjacent regions. The body-part regions are defined by the supplied semantic segmentation map, whereas the soft-routing weights are learned end-to-end without explicit expert-assignment supervision. Furthermore, a fine-to-coarse feature alignment module bridges global structural cues and local detail representations, enhancing both connectivity and visual fidelity across body parts. Experiments on DeepFashion at resolutions up to 512 × 256 and additional evaluation on SHHQ demonstrate improved visual realism, structural consistency, and quantitative performance under the evaluated settings. © 2026 Elsevier B.V.