This uses binary activations (i.e., 0 or 1) a simple Hebbian learning rule with binary weights: a weight is either 0 (does not exist) or is set to 1 if the pre- and post-synaptic activations are both 1. Once set to 1, it remains 1 forever. The architecture of a Willshaw network has two layers: an input layer that feeds forward to an output layer. All output nodes (artificial neurons) are updated simultaneously. The net input to each node is divided by the number of activated nodes in the input layer, where integer division is used (e.g., 7/5 = 1, not 1.4; the 0.4 is discarded; also 3/5 = 0, etc.). One could view this as a form of feedforward inhibition: The neurons in the input layer have an inhibitory effect on the output activations. This tends to normalize the output activations. Press for source code details in JavaScript.
Scroll down to see two videos that will help you complete the exercises.
The panel below has a Willshaw network and the pattern set demonstrated in Lecture 1 (see Powerpoint Presentation). Press the button. This will create three patterns, identical to those used in the lectures and named simply 1, 2, and 3. When pattern is pressed, you will see that Pattern 1 is activated on the (lower) input and (upper) output layer. Now, press to train the weights for this input-output pattern mapping.
You can verify the learned weights by checking Connection View (i.e., by clicking on the check box to the left of Connection View). If you now click any of the nodes in the output layer, you will see the weight values from each node in the input layer. There are not weights from the neighboring nodes in the output layer, because these connections do not exist in Willshaw networks.
Now repeat this for the other two nodes.
After having learned all patterns in Exercise 1, activate pattern 1 (press) and click on any black output nodes in the output layer. This sets their activation value from 1 (black) to 0 (white). Repeat that until the output layer is completely clear.
Now press once. You should see the correct, learned output pattern appear in the output layer. Use the dropdown menu attached to the output layer, which is showing Act, and switch it to Net. The black and weight activation values change to green and white net input values. By holding the cursor above a node, you will get a tooltip with the value. You can also check Show Values in the Settings panel. As you can see, the net inputs vary from 0 to 2 to 3. Dividing them by the number of activated input nodes, which is 3, has reduced these values to activations of 0 and 1.
Like in the lecture, repeat Exercise 3 but now with one (or more) input activations in the input layer set from 1 to 0. Is the recall still flawless? What about Patterns 2 and 3?
Repeat Exercise 2 but now lesion the weights with 50% by clicking once. Is the network still able to retrieve all patterns without error? Expect at least a few small errors to appear. You may set the Random Fraction to other values to lesion more or less, e.g., to 0.25 if you want to remove only 25% of the weights. This will be reflected in the Lesion button (e.g., it will read).
This is network is included if you want to play around more with Willshaw networks, e.g., with 2D patterns like Face-House. You may also store your own patterns by checking Store on the Pattern Menu and then clicking the pattern into which you want to store the current activations. By sure to not forget to check Present again after this. If you create your own patterns you may change their names for easy reference. Remember, however, that patterns sets cannot be stored permanently; you will loose them as soon as you move away from the webpage.