基于新型分析方法扩展加速扩散模型的目标分布

Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach

摘要 Abstract

加速扩散模型有望显著提高标准扩散过程的效率。理论上,这些模型已经显示出比普通扩散模型的标准$\mathcal O(1/\epsilon^2)$收敛率更快的收敛速度,其中$\epsilon$表示目标精度。然而,目前的理论研究仅在对目标分布类施加了平滑条件或有界支持等限制条件下证明了加速优势。在这项工作中,我们通过一种新的加速随机DDPM采样器显著扩展了目标分布类。具体而言,我们证明其对于之前未考虑的三大类广泛分布实现了加速性能。我们的第一类分布仅需对目标密度$q_0$施加平滑条件,这比现有沿整个采样路径对所有$q_t$施加的平滑条件更为宽松。我们的第二类分布仅需有限的二阶矩条件,允许的目标分布类远比现有的有限支撑条件宽泛。我们的第三类分布为高斯混合分布,对此我们的结果首次建立了加速保证。此外,在针对有界支撑分布的加速DDPM类型采样器中,我们的结果展示了对数据维度$d$依赖性的改进。我们的分析通过构建收敛误差的倾斜因子表示,并利用Tweedie公式处理泰勒展开项,引入了一种新颖的技术来建立性能保证。这一新的分析框架可能具有独立的研究兴趣。

Accelerated diffusion models hold the potential to significantly enhance the efficiency of standard diffusion processes. Theoretically, these models have been shown to achieve faster convergence rates than the standard $\mathcal O(1/\epsilon^2)$ rate of vanilla diffusion models, where $\epsilon$ denotes the target accuracy. However, current theoretical studies have established the acceleration advantage only for restrictive target distribution classes, such as those with smoothness conditions imposed along the entire sampling path or with bounded support. In this work, we significantly broaden the target distribution classes with a new accelerated stochastic DDPM sampler. In particular, we show that it achieves accelerated performance for three broad distribution classes not considered before. Our first class relies on the smoothness condition posed only to the target density $q_0$, which is far more relaxed than the existing smoothness conditions posed to all $q_t$ along the entire sampling path. Our second class requires only a finite second moment condition, allowing for a much wider class of target distributions than the existing finite-support condition. Our third class is Gaussian mixture, for which our result establishes the first acceleration guarantee. Moreover, among accelerated DDPM type samplers, our results specialized for bounded-support distributions show an improved dependency on the data dimension $d$. Our analysis introduces a novel technique for establishing performance guarantees via constructing a tilting factor representation of the convergence error and utilizing Tweedie's formula to handle Taylor expansion terms. This new analytical framework may be of independent interest.

基于新型分析方法扩展加速扩散模型的目标分布 - arXiv