Temporal prediction is essential in scientific and engineering problems, particularly for low-power online inference. Reservoir computing is attractive because a fixed recurrent system maps inputs into a high-dimensional state decoded by a simple readout. Liquid state machines instantiate this idea with recurrent spiking neurons, yielding sparse event-driven states suited to neuromorphic and digital hardware.
However, the stepwise protocol trains the readout from an instantaneous state, which can be information-limited when evidence is distributed across time. We improve temporal prediction by providing the readout with a finite temporal context. We concatenate a sliding window of recent reservoir spike states and train on either ridge regression or a transformer readout.
On the Nonlinear AutoRegressive Moving Average (NARMA5) benchmark, increasing context length improves both readouts. Median test NRMSE for ridge regression decreases from 2.106 under stepwise decoding to 0.317, an 85% reduction. The transformer readout shows the same trend, decreasing from 0.807 to 0.179, a 78% reduction.
The same qualitative trend is further observed on NARMA10 and Mackey Glass prediction tasks. These results demonstrate that exposing recent reservoir evolution at the decoder can substantially reduce the limitations of stepwise decoding while retaining the standard reservoir computing protocol. Reservoir computing (RC) provides a simple but powerful paradigm for processing temporal data by using a simple trainable readout layer to interpret the behaviour of a fixed dynamical system1.
Among different forms of RC, the liquid state machine (LSM) uses a biologically inspired spiking neural network (SNN) as the reservoir, instead of the continuous valued units used in classical echo state networks (ESNs)2,3. In an LSM, neurons represent and transmit information through discrete and sparse spike events. This event-driven representation aligns naturally with temporal coding in many real-world signals and can improve energy efficiency on neuromorphic or other event-driven hardware by reducing unnecessary activity and data movement for sparse spike workloads4,5.
Although recurrent SNNs can now be trained using gradient-based methods such as surrogate gradients6, reservoir computing is still attractive due to its simplicity and its practical task adaptation procedure. This makes it useful to distinguish between two ways of using recurrent spiking systems. In one case, the recurrent spiking network itself is treated as the trainable model, so task adaptation requires optimising internal synaptic weights or neuronal parameters.
Extract — continue reading at the source.
