算法稳定性是否可测试?基于计算约束的统一框架

Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints

摘要 Abstract

算法稳定性是学习理论中的核心概念,用于量化算法对训练数据微小变化的敏感程度。如果一个学习算法满足某些稳定性属性,则会带来许多重要的下游推论,如泛化能力、鲁棒性和可靠的预测推理。因此,验证特定算法是否具有稳定性是一个重要且实用的问题。然而,近期的研究结果表明,在未知分布下有限数据的情况下,对于数据位于不可数无限空间(如实值数据)的情形,测试黑盒算法的稳定性是不可能的。在本文中,我们将这一问题扩展到更广泛的场景,例如数据可能位于任何空间——例如分类数据。我们开发了一个统一的框架来量化测试算法稳定性的难度,该框架表明在所有情况下,如果可用数据有限,则穷举搜索本质上是唯一普遍有效的认证算法稳定性机制。由于在实际应用中,任何稳定性测试都会自然受到计算约束的影响,而穷举搜索是不可行的,这表明我们在测试黑盒算法的稳定性属性方面存在根本性的限制。

Algorithmic stability is a central notion in learning theory that quantifies the sensitivity of an algorithm to small changes in the training data. If a learning algorithm satisfies certain stability properties, this leads to many important downstream implications, such as generalization, robustness, and reliable predictive inference. Verifying that stability holds for a particular algorithm is therefore an important and practical question. However, recent results establish that testing the stability of a black-box algorithm is impossible, given limited data from an unknown distribution, in settings where the data lies in an uncountably infinite space (such as real-valued data). In this work, we extend this question to examine a far broader range of settings, where the data may lie in any space -- for example, categorical data. We develop a unified framework for quantifying the hardness of testing algorithmic stability, which establishes that across all settings, if the available data is limited then exhaustive search is essentially the only universally valid mechanism for certifying algorithmic stability. Since in practice, any test of stability would naturally be subject to computational constraints, exhaustive search is impossible and so this implies fundamental limits on our ability to test the stability property for a black-box algorithm.

算法稳定性是否可测试?基于计算约束的统一框架 - arXiv