模拟尖峰神经网络CoLaNET的数字机器学习算法

A Digital Machine Learning Algorithm Simulating Spiking Neural Network CoLaNET

摘要 Abstract

在过去的几年里,我们的研究团队致力于开发一种尖峰神经网络(Spiking Neural Network, SNN)架构,该架构可用于广泛的有监督学习分类任务。其运行条件是所有参与信号(被分类对象描述、正确类别标签和SNN决策)都具有尖峰特性。最终,我们发明了CoLaNET(列状分层网络)SNN架构。该架构的独特之处在于结合了对应于不同类别的原型网络结构以及显著区分同一类别实例的结构(=列)和列内功能不同的神经元群体(=层)。另一个独特特征是一种新颖的反-Hebbian可塑性和多巴胺调控可塑性的组合。尽管CoLaNET相对简单,但它包含多个超参数,针对具体分类任务选择这些超参数并非易事。此外,被分类数据的具体特征(例如MNIST数据集中的单独图片分类与连续视频流中物体分类)需要对CoLaNET结构进行特定修改。为了解决这些问题,必须对CoLaNET进行深入的数学探索。然而,由于SNN是随机离散系统,通常很难进行精确的数学分析。为此,我开发了一种连续数值(非尖峰)机器学习算法,可以以令人满意的精度近似CoLaNET的行为。本文对此进行了描述。目前,该算法正通过精确的解析方法进行研究。我们希望这项研究的结果能够应用于直接计算CoLaNET的超参数并优化其结构。

During last several years, our research team worked on development of a spiking neural network (SNN) architecture, which could be used in the wide range of supervised learning classification tasks. It should work under the condition, that all participating signals (the classified object description, correct class label and SNN decision) should have spiking nature. As a result, the CoLaNET (columnar layered network) SNN architecture was invented. The distinctive feature of this architecture is a combination of prototypical network structures corresponding to different classes and significantly distinctive instances of one class (=columns) and functionally differing populations of neurons inside columns (=layers). The other distinctive feature is a novel combination of anti-Hebbian and dopamine-modulated plasticity. While CoLaNET is relatively simple, it includes several hyperparameters. Their choice for particular classification tasks is not trivial. Besides that, specific features of the data classified (e.g. classification of separate pictures like in MNIST dataset vs. classifying objects in a continuous video stream) require certain modifications of CoLaNET structure. To solve these problems, the deep mathematical exploration of CoLaNET should be carried out. However, SNNs, being stochastic discrete systems, are usually very hard for exact mathematical analysis. To make it easier, I developed a continuous numeric (non-spiking) machine learning algorithm which approximates CoLaNET behavior with satisfactory accuracy. It is described in the paper. At present, it is being studied by exact analytic methods. We hope that the results of this study could be applied to direct calculation of CoLaNET hyperparameters and optimization of its structure.

模拟尖峰神经网络CoLaNET的数字机器学习算法 - arXiv