动态视图渲染的可学习无穷阶泰勒高斯方法

Learnable Infinite Taylor Gaussian for Dynamic View Rendering

摘要 Abstract

捕捉高斯分布的位置、旋转和尺度等属性随时间演化的特性是一项具有挑战性的任务,由于大量随时间变化的参数以及有限的光度数据,通常会导致收敛问题,难以找到最优解。虽然将所有输入送入端到端神经网络可以有效建模复杂的时序动态,但这种方法缺乏显式监督且难以生成高质量的变换场。另一方面,利用时序条件多项式函数建模高斯轨迹和方向提供了更明确且可解释的解决方案,但需要大量手工设计的工作,并且在不同场景中的泛化能力较弱。为克服这些局限性,本文提出了一种基于可学习无穷阶泰勒公式的新型方法来建模高斯分布的时序演化。该方法兼具基于隐式网络方法的灵活性和显式多项式函数的可解释性,能够在各种动态场景中实现更鲁棒和更具泛化性的高斯动力学建模。在公开数据集上的动态新视角渲染任务中进行了广泛的实验,结果表明所提出的方法在该领域达到了最先进的性能。更多详细信息请访问我们的项目页面(https://ellisonking.github.io/TaylorGaussian)。

Capturing the temporal evolution of Gaussian properties such as position, rotation, and scale is a challenging task due to the vast number of time-varying parameters and the limited photometric data available, which generally results in convergence issues, making it difficult to find an optimal solution. While feeding all inputs into an end-to-end neural network can effectively model complex temporal dynamics, this approach lacks explicit supervision and struggles to generate high-quality transformation fields. On the other hand, using time-conditioned polynomial functions to model Gaussian trajectories and orientations provides a more explicit and interpretable solution, but requires significant handcrafted effort and lacks generalizability across diverse scenes. To overcome these limitations, this paper introduces a novel approach based on a learnable infinite Taylor Formula to model the temporal evolution of Gaussians. This method offers both the flexibility of an implicit network-based approach and the interpretability of explicit polynomial functions, allowing for more robust and generalizable modeling of Gaussian dynamics across various dynamic scenes. Extensive experiments on dynamic novel view rendering tasks are conducted on public datasets, demonstrating that the proposed method achieves state-of-the-art performance in this domain. More information is available on our project page(https://ellisonking.github.io/TaylorGaussian).

动态视图渲染的可学习无穷阶泰勒高斯方法 - arXiv