摘要 Abstract
我们提出了一种全局可解释性方法,用于分析我们基于真实世界多任务卷积神经网络(MTCNN)的深度弃权分类器(DAC)在癌症病理报告自动标注任务中的组织学预测错误来源。我们的分类器在104万份手工标注样本上进行了训练和评估,能够同时对每份报告预测癌症部位、亚部位、组织学类型、侧别及行为特征。DAC框架允许模型在模棱两可的报告或混淆类别上选择弃权,以达到保留样本(未弃权样本)的目标准确率,但代价是覆盖范围降低。要求组织学任务达到97%准确率时,我们的模型仅保留了22%的样本,主要为较不模糊且常见的类别。通过GradInp技术进行局部可解释性分析,提供了高效获取数千个个体预测上下文推理的方法。我们的方法通过对约13000个局部解释进行降维处理,实现了对错误来源的全局识别,包括类别间的层次复杂性、标签噪声、信息不足以及冲突证据等问题。这表明可以通过排除标准、聚焦标注以及对涉及层次相关类别的错误减少惩罚等策略,迭代改进我们在这一复杂实际应用中的DAC模型。
We present a global explainability method to characterize sources of errors in the histology prediction task of our real-world multitask convolutional neural network (MTCNN)-based deep abstaining classifier (DAC), for automated annotation of cancer pathology reports from NCI-SEER registries. Our classifier was trained and evaluated on 1.04 million hand-annotated samples and makes simultaneous predictions of cancer site, subsite, histology, laterality, and behavior for each report. The DAC framework enables the model to abstain on ambiguous reports and/or confusing classes to achieve a target accuracy on the retained (non-abstained) samples, but at the cost of decreased coverage. Requiring 97% accuracy on the histology task caused our model to retain only 22% of all samples, mostly the less ambiguous and common classes. Local explainability with the GradInp technique provided a computationally efficient way of obtaining contextual reasoning for thousands of individual predictions. Our method, involving dimensionality reduction of approximately 13000 aggregated local explanations, enabled global identification of sources of errors as hierarchical complexity among classes, label noise, insufficient information, and conflicting evidence. This suggests several strategies such as exclusion criteria, focused annotation, and reduced penalties for errors involving hierarchically related classes to iteratively improve our DAC in this complex real-world implementation.