BackMix:通过去除前景-背景先验正则化开放集识别
BackMix: Regularizing Open Set Recognition by Removing Underlying Fore-Background Priors
摘要 Abstract
开放集识别(OSR)要求模型在对已知样本进行分类的同时检测未知样本,以满足实际应用需求。现有研究表明,利用辅助数据集中未知样本对OSR模型进行正则化取得了显著进展,但证明了其对选择此类已知异常值非常敏感。本文从一个新的视角讨论上述问题:我们能否在不精心挑选辅助已知异常值的情况下正则化OSR模型?我们首先从经验上和理论上探索前景和背景在开放集识别中的作用,并揭示以下两点:1)与前景相关的背景会误导模型并在遇到“部分”已知图像时导致失败;2)与前景无关的背景可以作为辅助的已知异常值并通过全局平均池化提供正则化。基于以上见解,我们提出了一种新方法——背景混合(BackMix),该方法通过将图像的前景与不同背景混合来去除潜在的前景-背景先验。具体而言,BackMix首先使用类激活图(CAMs)估计前景,然后随机用其他图像的背景替换图像补丁,以获得用于训练的混合图像。通过去除前景和背景的相关性,开放集识别性能得到了显著提高。该方法实现简单,在推理过程中无需额外操作,可以无缝集成到几乎所有现有的框架中。代码已发布在https://github.com/Vanixxz/BackMix。
Open set recognition (OSR) requires models to classify known samples while detecting unknown samples for real-world applications. Existing studies show impressive progress using unknown samples from auxiliary datasets to regularize OSR models, but they have proved to be sensitive to selecting such known outliers. In this paper, we discuss the aforementioned problem from a new perspective: Can we regularize OSR models without elaborately selecting auxiliary known outliers? We first empirically and theoretically explore the role of foregrounds and backgrounds in open set recognition and disclose that: 1) backgrounds that correlate with foregrounds would mislead the model and cause failures when encounters 'partially' known images; 2) Backgrounds unrelated to foregrounds can serve as auxiliary known outliers and provide regularization via global average pooling. Based on the above insights, we propose a new method, Background Mix (BackMix), that mixes the foreground of an image with different backgrounds to remove the underlying fore-background priors. Specifically, BackMix first estimates the foreground with class activation maps (CAMs), then randomly replaces image patches with backgrounds from other images to obtain mixed images for training. With backgrounds de-correlated from foregrounds, the open set recognition performance is significantly improved. The proposed method is quite simple to implement, requires no extra operation for inferences, and can be seamlessly integrated into almost all of the existing frameworks. The code is released on https://github.com/Vanixxz/BackMix.