面向无冲突训练的Physics Informed Neural Networks方法:ConFIG
ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks
摘要 Abstract
许多学习问题中的损失函数包含多个加项,这些加项可能产生冲突并导致更新方向不一致。对于Physics-Informed Neural Networks(PINNs),初始/边界条件和物理方程上的损失项尤为关键,因其任务难度较高。为改进PINNs提出的具有挑战性的多目标学习任务,我们提出了ConFIG方法,通过确保最终更新与每个特定损失梯度之间的点积为正,实现无冲突更新。该方法还保证了所有损失项的一致优化速率,并根据冲突水平动态调整梯度大小。此外,我们利用动量加速不同损失项的反向传播过程。我们提供了ConFIG方法收敛性的数学证明,并在一系列具有挑战性的PINNs场景中对其进行了评估。结果表明,ConFIG方法在性能和运行时间上均优于基线方法。我们还在经典多任务基准测试中验证了该方法,结果显示其表现极为出色。源代码可在https://tum-pbs.github.io/ConFIG获取。
The loss functions of many learning problems contain multiple additive terms that can disagree and yield conflicting update directions. For Physics-Informed Neural Networks (PINNs), loss terms on initial/boundary conditions and physics equations are particularly interesting as they are well-established as highly difficult tasks. To improve learning the challenging multi-objective task posed by PINNs, we propose the ConFIG method, which provides conflict-free updates by ensuring a positive dot product between the final update and each loss-specific gradient. It also maintains consistent optimization rates for all loss terms and dynamically adjusts gradient magnitudes based on conflict levels. We additionally leverage momentum to accelerate optimizations by alternating the back-propagation of different loss terms. We provide a mathematical proof showing the convergence of the ConFIG method, and it is evaluated across a range of challenging PINN scenarios. ConFIG consistently shows superior performance and runtime compared to baseline methods. We also test the proposed method in a classic multi-task benchmark, where the ConFIG method likewise exhibits a highly promising performance. Source code is available at https://tum-pbs.github.io/ConFIG