Diffusion Probabilistic Models: From Mathematical Foundations to Applications in Visual Synthesis and Restoration

Abstract

Diffusion probabilistic models have become a central framework for modern generative modeling, combining rigorous stochastic-process foundations with strong empirical performance across synthesis and restoration tasks. This thesis investigates diffusion models from mathematical principles to practical systems for visual synthesis and restoration, with emphasis on evaluation, controllable temporal generation, realistic image restoration, and efficient adaptation of large pretrained models.

The first contribution is EvalCrafter, a comprehensive evaluation framework for text-to-video generation. By analyzing 600,000 real-world prompts and curating a representative benchmark of 700 prompts, EvalCrafter organizes evaluation into four dimensions: visual quality, text-video alignment, motion quality, and temporal consistency. Extensive human-preference studies identify which automatic metrics correlate with subjective judgment and establish a more reliable basis for benchmarking video generation models.

The second contribution addresses SRRIIE: super-resolving real-world image illumination enhancement under realistic degradations. We introduce a paired real-world dataset for this task and develop a conditional diffusion model with Retinex-inspired modeling and a Time-Melding Condition (TMC) mechanism. The proposed method allows resolution recovery and illumination correction to be optimized within a unified reverse process. Experiments on synthetic and real low-light images show that this joint design improves both quantitative fidelity and perceptual quality.

The third contribution is the Frame-Aware Video Diffusion Model (FVDM), which provides the thesis's main theoretical development by replacing the conventional scalar timestep with a Vectorized Timestep Variable (VTV). This formulation allows different video frames to evolve under different noise levels within one diffusion trajectory. We derive the corresponding stochastic framework, establish the reverse-time formulation, and introduce the Probabilistic Timestep Sampling Strategy (PTSS) to make training tractable. The resulting model supports text-to-video generation, image-to-video synthesis, interpolation, and video extension within a unified framework.

The fourth contribution is Pusa V1.0, which translates the vectorized-timestep perspective to industrial-scale image-to-video generation through Vectorized Timestep Adaptation (VTA). By adapting pretrained large-scale video diffusion models rather than retraining them from scratch, Pusa achieves state-of-the-art VBench-I2V performance with about 4,000 training samples while requiring only a small fraction of the cost associated with task-specific alternatives. This demonstrates that the theoretical gains of vectorized timesteps can be realized in practical, efficient systems.

Taken together, these contributions show how diffusion models can be advanced coherently across theory, evaluation, restoration, and deployment. The thesis contributes rigorous mathematical foundations for vectorized temporal diffusion, more faithful evaluation protocols for video synthesis, a strong restoration framework for realistic image degradation, and an efficient adaptation strategy for large video generation models. In combination, these results strengthen diffusion probabilistic models as a flexible foundation for visual synthesis and restoration.

Date of Award	24 Jun 2026
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Roy HE (Supervisor) & Dinggang Shen (External Co-Supervisor)