Spiking Neural Networks (SNNs)

Spiking neural networks are the computational models behind neuromorphic computing. Unlike ANNs that use continuous activations, SNNs communicate through discrete spikes over time.

How SNNs Work

In a spiking neural network, each neuron accumulates incoming signals over time in its membrane potential. When this potential exceeds a threshold, the neuron emits a spike (a binary event) and resets. This is fundamentally different from artificial neural networks where every neuron produces a continuous output on every forward pass.

Neuron Models

The LIF Neuron

The Leaky Integrate-and-Fire neuron is the workhorse of practical SNNs:

# LIF neuron dynamics (discrete time)
# V[t] = beta * V[t-1] + W @ X[t]  (leak + integration)
# S[t] = 1 if V[t] >= threshold else 0  (spike)
# V[t] = V[t] * (1 - S[t])  (reset after spike)

import snntorch as snn
import torch

# Create a LIF neuron layer
lif = snn.Leaky(beta=0.9, threshold=1.0)
mem = lif.init_leaky()  # initialize membrane potential

# Process input over time
spikes = []
for t in range(num_steps):
    spk, mem = lif(input_data[t], mem)
    spikes.append(spk)

Spike Encoding

To feed data into an SNN, you must convert it to spikes:

• Rate coding: Higher values produce more frequent spikes. Simple but loses temporal precision.
• Temporal coding: Values encoded in spike timing. Higher values spike earlier. More efficient.
• Delta coding: Spikes encode changes in input. Ideal for event-driven sensors.
• Population coding: Multiple neurons with overlapping receptive fields represent each value.

Training SNNs

The non-differentiable spike function creates a challenge for gradient-based training. Solutions include:

1. Surrogate Gradients

   Replace the non-differentiable spike function with a smooth approximation during backpropagation. This is the most popular approach.

2. ANN-to-SNN Conversion

   Train a standard ANN, then convert weights and activations to spiking equivalents. Loses some temporal advantages.

3. STDP (Spike-Timing Dependent Plasticity)

   Biologically inspired local learning rule. Synapses strengthen when pre-synaptic spikes precede post-synaptic spikes.

4. Evolutionary Methods

   Use genetic algorithms or neuroevolution to optimize SNN architectures and parameters.

SNN Frameworks

• snnTorch: PyTorch-based. Best for deep learning practitioners. Surrogate gradient training.
• Norse: PyTorch-based. Focuses on biologically plausible models and recurrent SNNs.
• BindsNET: Biologically inspired. STDP and reward-modulated learning rules.
• Lava: Intel's framework for Loihi. Hardware-aware SNN development.
• NEST: Large-scale neuroscience simulator. Millions of neurons.