KAC: 用于持续学习的Kolmogorov-Arnold分类器

KAC: Kolmogorov-Arnold Classifier for Continual Learning

摘要 Abstract

持续学习要求模型能够在连续的任务之间不断训练且不遗忘先前的知识。大多数现有方法采用线性分类器,但线性分类器在学习新任务时难以保持稳定的分类空间。受Kolmogorov-Arnold网络(KAN)在简单持续回归任务中保持学习稳定性成功经验的启发,我们探索其在更复杂持续学习场景中的潜力。本文介绍了一种基于KAN结构为持续学习设计的新型分类器——Kolmogorov-Arnold分类器(KAC)。我们深入研究了KAN的样条函数的影响,并引入径向基函数(RBF)以提高其与持续学习的兼容性。我们将线性分类器替换为KAC,并在多个近期方法中进行实验,涉及各种持续学习基准,所有实验均显示性能提升,凸显了KAC在持续学习中的有效性和鲁棒性。代码可在https://github.com/Ethanhuhuhu/KAC获取。

Continual learning requires models to train continuously across consecutive tasks without forgetting. Most existing methods utilize linear classifiers, which struggle to maintain a stable classification space while learning new tasks. Inspired by the success of Kolmogorov-Arnold Networks (KAN) in preserving learning stability during simple continual regression tasks, we set out to explore their potential in more complex continual learning scenarios. In this paper, we introduce the Kolmogorov-Arnold Classifier (KAC), a novel classifier developed for continual learning based on the KAN structure. We delve into the impact of KAN's spline functions and introduce Radial Basis Functions (RBF) for improved compatibility with continual learning. We replace linear classifiers with KAC in several recent approaches and conduct experiments across various continual learning benchmarks, all of which demonstrate performance improvements, highlighting the effectiveness and robustness of KAC in continual learning. The code is available at https://github.com/Ethanhuhuhu/KAC.

KAC: 用于持续学习的Kolmogorov-Arnold分类器 - arXiv