Glivenko-Cantelli定理在$f$-散度中的推广

Glivenko-Cantelli for $f$-divergence

摘要 Abstract

我们将著名的Glivenko-Cantelli定理(有时称为统计学的基本定理)从其标准设定下的总变差距离推广到所有的$f$-散度。这一努力的关键障碍在于如何在一个$\sigma$-代数的子集上定义$f$-散度,该子集构成一个$\pi$-系统但不是$\sigma$-子代数。这是我们工作的次要贡献之一。我们将证明这种在射线$\pi$-系统上的$f$-散度概念几乎保留了标准$f$-散度的所有已知性质,得到了Kolmogorov-Smirnov距离的一种新积分表示,并具有Glivenko-Cantelli定理。我们还将讨论$f$-散度的Vapnik-Chervonenkis理论的可能性。

We extend the celebrated Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, from its standard setting of total variation distance to all $f$-divergences. A key obstacle in this endeavor is to define $f$-divergence on a subcollection of a $\sigma$-algebra that forms a $\pi$-system but not a $\sigma$-subalgebra. This is a side contribution of our work. We will show that this notion of $f$-divergence on the $\pi$-system of rays preserves nearly all known properties of standard $f$-divergence, yields a novel integral representation of the Kolmogorov-Smirnov distance, and has a Glivenko-Cantelli theorem. We will also discuss the prospects of a Vapnik-Chervonenkis theory for $f$-divergence.