推进赫布卷积神经网络的生物合理性与有效性

Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks

摘要 Abstract

本文研究了将赫布学习整合到卷积神经网络(CNNs)中用于图像处理的方法,系统地探索了不同的架构以构建最优配置,同时保持生物学上的合理性。赫布学习基于局部无监督的神经信息形成特征表示,为流行的但可能在生物学上不合理且计算密集的反向传播学习算法提供了一种替代方案。提出的最优架构显著提升了近期关于将赫布学习与竞争机制及CNN结合的研究成果,通过在一个单一模型中引入硬胜者通吃(WTA)竞争、高斯侧抑制机制以及Bienenstock-Cooper-Munro(BCM)学习规则,扩展了其表征能力。在CIFAR-10测试集后半部分的最后几个epoch中,所得最优模型的平均分类准确率达到了75.2%,这一结果与端到端反向传播变体相当,并且比相同网络深度的CNN中当前最先进的硬-WTA性能(64.6%)高出10.6%。此外,该模型在MNIST(98%)和STL-10(69.5%)数据集上也表现出竞争力。结果还显示,通过越来越复杂和抽象的感受野,实现了清晰的分层稀疏学习。总之,我们的实现不仅提高了学习表示的性能和泛化能力,而且构成了迈向更生物现实的人工神经网络的重要一步。

The research presented in this paper advances the integration of Hebbian learning into Convolutional Neural Networks (CNNs) for image processing, systematically exploring different architectures to build an optimal configuration, adhering to biological tenability. Hebbian learning operates on local unsupervised neural information to form feature representations, providing an alternative to the popular but arguably biologically implausible and computationally intensive backpropagation learning algorithm. The suggested optimal architecture significantly enhances recent research aimed at integrating Hebbian learning with competition mechanisms and CNNs, expanding their representational capabilities by incorporating hard Winner-Takes-All (WTA) competition, Gaussian lateral inhibition mechanisms, and Bienenstock-Cooper-Munro (BCM) learning rule in a single model. Mean accuracy classification measures during the last half of test epochs on CIFAR-10 revealed that the resulting optimal model matched its end-to-end backpropagation variant with 75.2% each, critically surpassing the state-of-the-art hard-WTA performance in CNNs of the same network depth (64.6%) by 10.6%. It also achieved competitive performance on MNIST (98%) and STL-10 (69.5%). Moreover, results showed clear indications of sparse hierarchical learning through increasingly complex and abstract receptive fields. In summary, our implementation enhances both the performance and the generalisability of the learnt representations and constitutes a crucial step towards more biologically realistic artificial neural networks.

推进赫布卷积神经网络的生物合理性与有效性 - arXiv