Towards Mathematical Foundations of Diffusion Models

Diffusion models are a powerful generative AI technology that has made significant strides in areas like computer vision, natural language processing, and computational biology. However, despite their empirical success, the mathematical understanding of diffusion models is still underdeveloped. This lack of theoretical grounding may limit further innovation and prevent the models from reaching their full potential. This research seeks to address these theoretical gaps, with a dual focus: (1) to enhance the computational efficiency of diffusion models by improving their convergence rates, and (2) to deepen the theoretical understanding of their robustness, especially when applied to datasets with challenges such as class imbalances, outliers, and non-Gaussian characteristics. Specifically, the project will concentrate on two main directions. First, we propose novel frameworks, along with new discretization schemes and parallelization techniques, to boost the efficiency of diffusion models while ensuring rigorous performance guarantees. Second, we plan to explore the behavior of diffusion models under heavy-tailed noise distributions and provide theoretical guarantees regarding their robustness and reliability. It is important to note that diffusion models are inherently related to sampling, as they generate samples by progressively transforming noise into data through a series of stochastic steps. In essence, diffusion models rely on the process of sampling from complex distributions, with the model's efficiency and accuracy closely tied to the effectiveness of the underlying sampling techniques. Building on the PI’s previous work in sampling theory, particularly in collaboration with Arnak Dalalyan on log-concave sampling, this research continues efforts to deepen the theoretical understanding of sampling methods. Ultimately, this project aims to generate new insights that strengthen the mathematical foundations of diffusion models, enhancing their real-world applications and supporting further advancements in AI.