Connectionist Models: 

Neural Network Exercises

Jaap Murre, University of Amsterdam


Biological spiking neurons

Exercise 15: The Izhikevich Model Neuron

Open the 0verview page of twenty types of biological spiking neurons, all generated by the Izhikevich Model Neuron (2004, see his Figure 1).

15.a Open two pages with the overview page. In page 1 click on Neuron Type A and in page 2 click on B. There are several small differences in the parameters but there is a large difference in the input current. This suggests that if you lower the input current in Neuron Type A, you may also get a single spike. Lower the current in A until you indeed get a single spike. Ensure that there is no second spike outside the graph by setting T (the total time of X-axis in ms) to 10000. What is the current you finally got? What would be a function of this type of neuron in terms of information processing in the brain?

15.b Open a page with Neuron Type G (Class 1 Excitable). Observe that the X-axis is 300 ms (settable with the T parameter). What is roughly the frequency of the first inter-spike interval (in spike/s or Hz)? And of the last? What could be an important function of this type of neuron?

15.c Open a page with Neuron Type K (Resonator). Play with parameter a in steps of 0.01 (a few steps up and down). Can you find a value where the neuron spikes for the first current pair but not the second? What happens when you increase all current pulses to 0.80 mA? With these higher currents, try increasing a in steps of 0.01. What happens then? What would be the use of neurons of this type (again, in terms of information processing)? What would be a limiting factor in its use?

15.d Open a page with Neuron Type P (Bistability). The first pulse causes tonic spiking, which is abolished at the second pulse. This pulse must arrive in the current point of the phase of the spike. Verify this by trying an interval of 176 instead of 174 ms. A possible use for this mechanism would be a type of short-term memory: one pulse turns on the memory, which is represented by the firing neuron, another spike turns off the memory when it is no longer needed.

15.e Thus far in the course, we have worked with networks that had very simple neurons and yet seemed to be able to carry out almost any kind of information processing. Why do you think the real brain has so many different types of neurons if it could in principle do all its processing with just a single type (say, Type A)? To answer this question you may (but need not) take an evolutionary perspective.

Exercise 16: A connected groups of Izhikevich neurons

Open the webpage with a group of biological spiking neurons, based on the the Izhikevich Model Neuron (2003, see his Figure 3). When you press , a spike plot will be generated based on the parameters specified at the top of the page.

Depending on the speed of your computer, generation may take quite some time, especially with large groups. Most of this time is spent on generating the plot, not on the actual neural calculations. However, if you use a group size of 100 it should take only 10 to 20 seconds. If you have generated a plot and want to generate another one (e.g., with different parameters) either duplicate the page and work with the duplicate or reload the page! Due to a bug I have not been able to find, pressing for the second (or third) time, generates a plot that is faulty.

16.a Open three new pages with the neuron group simulator. Use the default values of the parameters. In the first page, change the current from 4 to 3, in the third page to 5. You now have currents of 3, 4, and 5. Generate the plots in pages 1 to 3. Describe the regularities in the spike patterns. How do the pages with higher current differ from the pages with lower current? Are the same neurons involved? Are more neurons firing? Are there neurons that stop firing with higher current?

16.b Repeat the previous assignment but now with (Clamped, Current) is (3,4), (4,3), and (6,2). (Be sure to reload the pages first!) What do you observe now? For bonus points: Even though the total current stays the same, the average number of firing neurons increase when more neurons are clamped (with a lower current of 2). Why do you think this is the case?

16.c Suppose you could feed the firing patterns of the non-clamped node into a backpropagation network, you could learn to recognize the input vector of clamped nodes (which nodes are clamped and with what current)? Why is this the case? What is different from a regular backpropagation network (think: 'time')?

16.d Repeat 16.a but now with size 100, 200, and 400. At size 400 you start to see rhythms in the summed spike plots. What would you expect with size 800 (try if you are brave!)? Why do you think you see these rhythms getting more prominent at larger sizes (with other parameters being kept equal)? Why does the real brain not have super strong rhythms even though it is very large? What would you need to do to ensure that the spike patterns (and hence the observed rhythms) are representative for the real brain?

16.e (For bonus points) It is almost impossible to prevent rhythms from occurring in large networks and as such you can call them emergent phenomena or even epiphenomena. What can be an important function of rhythms with respect to information transfer?