基于扩散模型的通用去噪合成方法GenDeg用于一体化图像恢复
GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration
摘要 Abstract
近年来,基于深度学习的一体化图像恢复(AIOR)模型取得了显著进展。然而,由于训练数据分布外样本的泛化能力较差,其实际应用受到限制。这一局限性主要源于现有数据集中退化变化和场景多样性不足,无法充分反映真实世界的情景。此外,采集大规模的真实配对数据(如雾霾、低光照和雨滴等退化情况)往往繁琐且有时不可行。本文利用潜在扩散模型的生成能力,从干净图像中合成高质量的退化图像。具体而言,我们提出了GenDeg,这是一种退化和强度感知的条件扩散模型,能够在干净图像上生成多样化的退化模式。通过GenDeg,我们在六种退化类型(雾霾、雨、雪、运动模糊、低光照和雨滴)中共合成超过55万样本。这些生成的样本与现有数据集结合,形成了包含超过75万样本的GenDS数据集。实验结果表明,基于GenDS数据集训练的图像恢复模型在分布外性能方面表现出显著提升,优于仅基于现有数据集训练的模型。此外,我们还提供了关于基于扩散模型的合成退化对AIOR影响的全面分析。
Deep learning-based models for All-In-One Image Restoration (AIOR) have achieved significant advancements in recent years. However, their practical applicability is limited by poor generalization to samples outside the training distribution. This limitation arises primarily from insufficient diversity in degradation variations and scenes within existing datasets, resulting in inadequate representations of real-world scenarios. Additionally, capturing large-scale real-world paired data for degradations such as haze, low-light, and raindrops is often cumbersome and sometimes infeasible. In this paper, we leverage the generative capabilities of latent diffusion models to synthesize high-quality degraded images from their clean counterparts. Specifically, we introduce GenDeg, a degradation and intensity-aware conditional diffusion model capable of producing diverse degradation patterns on clean images. Using GenDeg, we synthesize over 550k samples across six degradation types: haze, rain, snow, motion blur, low-light, and raindrops. These generated samples are integrated with existing datasets to form the GenDS dataset, comprising over 750k samples. Our experiments reveal that image restoration models trained on the GenDS dataset exhibit significant improvements in out-of-distribution performance compared to those trained solely on existing datasets. Furthermore, we provide comprehensive analyses on implications of diffusion model-based synthetic degradations for AIOR.