Ising Models for Neural Data

John Hertz, NBI and Nordita

Yasser Roudi, Nordita

Joanna Tyrcha, Mathematical Statistics, Stockholm University

Recent technological advances now make it possible to record simultaneously the activities of hundreds of neurons. However, it is not at all obvious how to interpret the huge numbers of multi-unit spike patterns that can now be measured. New mathematical methods and techniques are required in order to gain insight from such data into how neuronal networks function. We are exploring the use of Ising models to help do this. Although the use of Ising models for neural networks has a long history, going back to the Hopfield model and even earlier work, our approach is different from the classical one in that it is data-based: rather than invent a model and see how well it describes experiments, we start with experimental data and try to infer a model that fits them.

Equilibrium models

(Y Roudi, J Tyrcha, J Hertz, http://arxiv.org/abs/0902.2885)

Posters: Cosyne 2008: ppt pdf Society for Neuroscience 2008: ppt pdf

Slides for a seminar by J Hertz, Stockholm University, 26 March 2009: ppt pdf

In one simple approach, we take the time-binned spike records S_i(t) (= +1 if neuron i spikes in time bin t and -1 if it does not) and compute the equal-time means m_i = <S_i> and correlations C_ij = <S_iS_j> - <S_i><S_j>. The idea (first proposed by Schneidman et al) is to find an Ising model

that has the same means and correlations as the data. We call the J_ij functional connections. They are not the synaptic strengths in the real neural network that generated the data; they only explain the data in the sense that the Ising model with these parameters has the measured means and correlations.

Fitting the parameters

Boltzmann learning, in which one iteratively adjusts the J_ij according to

is guaranteed to give an Ising model with the right means and correlations, but it is very slow for large networks since the average with the current J requires long Monte Carlo runs at every iteration step. Therefore approximate methods are very useful. We have examined several such methods based on independent pair approximations, mean field methods from spin glass theory, and combinations thereof. One such method is based on the TAP equations

,

from which we find

,

which can be solved for J_ij. This is very fast to evaluate, even for large networks (as are all the approximate methods studied).

The figure below shows a comparison of the various approximations with the (numerically exact) Boltzmann learning results for a population of 200 neurons in a realistic cortical network simulation.

(a) is for a naive MFT in which Onsager reaction term in the TAP equations is ignored.

(b) is for an independent-pair approximation

(c) is for the low-rate limit of the independent-pair approximation

(d) is for the TAP approximation described above

(e) is for an a combination of the naive MFT and independent-pair approximations (Sessak and Monasson)

(f) is for the average of the TAP and Sessak-Monasson results

The TAP and Sessak-Monasson approximations are good, and their (ad hoc) average is even better. The other approximations, while they work well for small populations (N < 20 or so), are not good for large populations.

How good is the Ising model fit?

The approach above gives functional connections J_ij that explain the first and second-order statistics of the spike patterns. But it is not guaranteed to give higher-order statistics correctly. As a measure of how good an overall fit we make to the true distribution we calculate the Kullback-Leibler distance

.

For a goodness-of-fit measure we compare this with the KL distance between the data and an independent-neuron model (J_ij=0):

.

G (from our computations) is shown as a function of population size N in the following figure.

The Ising model is evidently a good fit for small N, but the quality deteriorates with N. For N ~ 100 or larger, it is getting higher-order statistics badly wrong.

The good fit at small N is an automatic thing if the firing rates are low enough (Nn << 1, where n is the average number of spikes per neuron per time bin; this means up to N ~ 10 for these data), as shown by Roudi et al, PLoS Comp Biol, in press, available at http://arxiv.org/abs/0811.0903.

Asymmetric models

The J_ij obtained by the above approach agree badly with the true synaptic couplings in the simulated cortical model network. This is possibly because synapses are directed (so J_ij is not in general equal to J_ji), while the equilibrium Ising model requires J_ij=J_ji. We are now exploring asymmetric Ising models of the form

with non-symmetric J_ij.

The graphs below show how a simple approximate algorithm, based on expanding around an independent-neuron model, works for a diluted random-J_ij network of 1000 units.

The left-hand panel shows the J_ij in the model, which was run for 100000 time steps. The connections extracted from these data are shown in the right-hand panel. Although the magnitudes are off, the synapses are clearly identified correctly.

---

Last updated 29 March 2009 by hertz@nordita.org