随机配置网络的时间序列数据分析递归模型

Recurrent Stochastic Configuration Networks for Temporal Data Analytics

摘要 Abstract

利用神经网络进行时间数据建模在许多领域应用广泛,包括时间序列预测和控制工程。本文旨在开发一种用于问题求解的随机配置网络递归版本(RSCNs),其中我们不对输入变量的动力学阶数做出假设。给定一组历史数据后,首先通过监督机制构建初始RSCN模型,并利用投影算法进行输出权重的在线更新。本文还建立了若干理论结果,包括回声状态特性、RSCNs在离线和在线学习中的通用逼近特性以及输出权重的收敛性。所提出的RSCN模型在输入随机权值矩阵的赋值方式及反馈矩阵的特殊结构方面显著区别于著名的回声状态网络(ESNs)。本文对长短期记忆网络(LSTM)、原始ESN及其几种最先进的ESN方法(如简单循环水库(SCR)、多项式ESN(PESN)、泄漏积分器ESN(LIESN)和RSCN)进行了全面比较研究。数值结果表明,所提出的RSCN在所有数据集上的表现均优于其他方法。

Temporal data modelling techniques with neural networks are useful in many domain applications, including time-series forecasting and control engineering. This paper aims at developing a recurrent version of stochastic configuration networks (RSCNs) for problem solving, where we have no underlying assumption on the dynamic orders of the input variables. Given a collection of historical data, we first build an initial RSCN model in the light of a supervisory mechanism, followed by an online update of the output weights by using a projection algorithm. Some theoretical results are established, including the echo state property, the universal approximation property of RSCNs for both the offline and online learnings, and the convergence of the output weights. The proposed RSCN model is remarkably distinguished from the well-known echo state networks (ESNs) in terms of the way of assigning the input random weight matrix and a special structure of the random feedback matrix. A comprehensive comparison study among the long short-term memory (LSTM) network, the original ESN, and several state-of-the-art ESN methods such as the simple cycle reservoir (SCR), the polynomial ESN (PESN), the leaky-integrator ESN (LIESN) and RSCN is carried out. Numerical results clearly indicate that the proposed RSCN performs favourably over all of the datasets.