基于关键点引导扩散模型的单样本飞机SAR到光学图像翻译

Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models

摘要 Abstract

合成孔径雷达(SAR)图像提供了全天候、全时段及高分辨率的成像能力,但其独特的成像机制使得解释高度依赖专家知识,限制了可解释性,特别是在复杂目标任务中。将SAR图像转换为光学图像是一种增强解释能力和支持下游任务的有前景的解决方案。大多数现有研究集中在场景级转换,由于缺乏配对数据以及准确保存轮廓和纹理细节的挑战,关于目标级转换的工作有限。为了解决这些问题,本研究提出了一种基于关键点引导扩散模型(KeypointDiff)的未配对飞机目标SAR到光学图像转换框架。该框架通过关键点引入目标类别和方位角的监督,并针对未配对数据设计了一种训练策略。在此基础上,基于无分类器引导的扩散架构,设计了一个类别-方位引导模块(CAGM),将类别和方位信息整合到扩散生成过程中。此外,采用对抗损失和一致性损失,提高图像保真度和细节质量,特别适用于飞机目标。在采样过程中,借助预训练的关键点检测器,该模型消除了手动标注类别和方位信息的需求,实现了自动化的SAR到光学图像转换。实验结果表明,所提出的方法在多个指标上优于现有方法,提供了一种高效且有效的目标级SAR到光学图像转换及其下游任务解决方案。此外,在关键点检测器的帮助下,该方法对未训练的飞机类型表现出强大的零样本泛化能力。

Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.

基于关键点引导扩散模型的单样本飞机SAR到光学图像翻译 - arXiv