Spiking Neural Networks have been called the 3rd generation of Artificial Neural Network - with the 1st being binary neurons of the Hopfield type with activation values of only 0 or 1, and the 2nd being sigmoidal model neurons with continuous real-valued activations approximating the 'firing rate' of real neurons, used in back-propagation and self-organising neural networks and their many variants.

Spiking neuron models are vital in the work being carried out in the Thinking Systems project because we are working closely with neuroscientists in the Queensland Brain Institute. Models of neurons using sigmoidal (or other) real-valued activation functions lack the biological fidelity required to incorporate knowledge from the neurosciences into our models.

Real neurons exhibit a very broad range of behaviours (tonic and phasic spiking, bursting, spike latency, spike frequency adaptation, resonance, threshold variability, input accommodation and bistability). It's unlikely that these behaviours have no computational significance. In a small network of only 100 neurons (80 excitatory, 20 inhibitory), simply varying the relative strengths of the excitatory and inhibitory connections can induce a wide range of dynamic behaviours. When this behaviour is measured and plotted over the "phase space", curious non-linear structures are often apparent, as in the graph below which measures the structure or information-carrying capacity of the network in its induced state. The four inserted sub-graphs in the figure show typical spike raster plots of the 100 neurons (Y-axis) against time (X-axis) over 1000 milliseconds for different relative connection strengths. For more information download TS Spiking.ppt.

One of the most exciting characteristics of spiking neural networks, with the potential to create a step-change in our knowledge of neural computation, is that they are innately embedded in time. Spike latencies, axonal conduction delays, refractory periods, neuron resonance and network oscillations all give rise to an intrinsic ability to process time-varying data in a more natural and computationally powerful way than is available to 2nd generation models. Real brains are embedded in a time-varying environment; almost all real-world data and human or animal mental processing has a temporal dimension. Evidence is growing that rhythmic brain oscillations are strongly connected to cognitive processing. So utilising Spiking Neural Networks may be one of the first steps needed to bridge the current divide between existing ANN models and more flexible, realistic and, dare I say, intelligent, behaviour from artificial systems.

Two questions are paramount:

1) What level of detail is required to be modelled in spiking neurons in order to sufficiently capture the computational abilities of nervous systems (or alternatively what level of abstraction is appropriate)?

2) What network parameters and topologies will support some of the properties observed in networks of real neurons?

Currently my research is focussed on answering these questions. Soon spiking neural models will be incorporated into the larger Thinking Systems project, which aims to discover and implement new and unique brain-inspired algorithms for navigating through both physical and conceptual spaces.

Last updated October 2007