DOF-GS:用于捕获后重新聚焦、散焦渲染和去模糊的可调景深三维高斯点投射框架
DOF-GS: Adjustable Depth-of-Field 3D Gaussian Splatting for Post-Capture Refocusing, Defocus Rendering and Blur Removal
摘要 Abstract
近年来,三维高斯点投射(3DGS)技术实现了高质量的三维场景重建和实时新视图合成。然而,这些方法受限于针孔相机模型且缺乏对散焦效应的有效建模。为此,我们提出了DOF-GS——一种基于新的3DGS框架,采用有限孔径相机模型并具有明确且可微分的散焦渲染功能,使其能够作为捕获后的控制工具。通过训练带有适度散焦模糊的多视图图像,DOF-GS学习内在的相机特性,并重建底层场景的清晰细节,特别是通过按需控制光圈和焦距实现不同景深效果的渲染。此外,我们的框架在优化过程中提取弥散圆线索,以识别输入视图中的对焦区域,从而增强重建的三维场景细节。实验结果表明,DOF-GS支持捕获后的重新聚焦、可调节的散焦以及从未经校准的散焦模糊的多视图图像中进行高质量的全对焦渲染。
3D Gaussian Splatting (3DGS) techniques have recently enabled high-quality 3D scene reconstruction and real-time novel view synthesis. These approaches, however, are limited by the pinhole camera model and lack effective modeling of defocus effects. Departing from this, we introduce DOF-GS--a new 3DGS-based framework with a finite-aperture camera model and explicit, differentiable defocus rendering, enabling it to function as a post-capture control tool. By training with multi-view images with moderate defocus blur, DOF-GS learns inherent camera characteristics and reconstructs sharp details of the underlying scene, particularly, enabling rendering of varying DOF effects through on-demand aperture and focal distance control, post-capture and optimization. Additionally, our framework extracts circle-of-confusion cues during optimization to identify in-focus regions in input views, enhancing the reconstructed 3D scene details. Experimental results demonstrate that DOF-GS supports post-capture refocusing, adjustable defocus and high-quality all-in-focus rendering, from multi-view images with uncalibrated defocus blur.