摘要 Abstract
我们研究了从任意分布中学习高密度区域的问题。给定目标覆盖参数$\delta$以及对任意分布$D$的样本访问权限,我们的目标是输出一个置信集$S\subset \mathbb{R}^d$,使得$S$达到$D$的$\delta$覆盖,即$\mathbb{P}_{y \sim D}[y \in S] \geq \delta$,并且$S$的体积尽可能小。这是高维统计学中的核心问题,具有在置信集构建、不确定性量化和支持估计中的应用。在最一般的情况下,该问题是统计上不可处理的,因此我们将注意力限制在与具有有界VC维的概念类$C$竞争。算法与类$C$竞争的条件是:给定来自任意分布$D$的样本,它能在多项式时间内输出一个实现$D$的$\delta$覆盖且其体积与具有所需覆盖$\delta$的最小集合相比具有竞争力的集合。即使在$C$为所有欧几里得球体集合的基本情况下,这个问题也是计算上具有挑战性的。现有的基于核的方法可以在多项式时间内找到一个体积为最佳球体体积的$\exp(\tilde{O}(d / \log d))$倍的球体。我们的主要结果是一种算法,它能找到一个置信集,其体积是具有所需覆盖的最优球体体积的$\exp(\tilde{O}(d^{2/3}))$倍。该算法是非适当的(输出一个椭球)。结合我们关于适当学习球体在体积上达到$\exp(\tilde{O}(d^{1-o(1)}))$近似因子的计算不可处理性结果,我们的研究结果在置信集的适当学习与(非适当)学习之间提供了一个有趣的分离。
We study the problem of learning a high-density region of an arbitrary distribution over $\mathbb{R}^d$. Given a target coverage parameter $\delta$, and sample access to an arbitrary distribution $D$, we want to output a confidence set $S \subset \mathbb{R}^d$ such that $S$ achieves $\delta$ coverage of $D$, i.e., $\mathbb{P}_{y \sim D} \left[ y \in S \right] \ge \delta$, and the volume of $S$ is as small as possible. This is a central problem in high-dimensional statistics with applications in finding confidence sets, uncertainty quantification, and support estimation. In the most general setting, this problem is statistically intractable, so we restrict our attention to competing with sets from a concept class $C$ with bounded VC-dimension. An algorithm is competitive with class $C$ if, given samples from an arbitrary distribution $D$, it outputs in polynomial time a set that achieves $\delta$ coverage of $D$, and whose volume is competitive with the smallest set in $C$ with the required coverage $\delta$. This problem is computationally challenging even in the basic setting when $C$ is the set of all Euclidean balls. Existing algorithms based on coresets find in polynomial time a ball whose volume is $\exp(\tilde{O}( d/ \log d))$-factor competitive with the volume of the best ball. Our main result is an algorithm that finds a confidence set whose volume is $\exp(\tilde{O}(d^{2/3}))$ factor competitive with the optimal ball having the desired coverage. The algorithm is improper (it outputs an ellipsoid). Combined with our computational intractability result for proper learning balls within an $\exp(\tilde{O}(d^{1-o(1)}))$ approximation factor in volume, our results provide an interesting separation between proper and (improper) learning of confidence sets.