This uses bipolar activations (i.e., -1 or 1) and a simple Hebbian learning rule. Press in the simulator pannel below for source code details in JavaScript.
N.B. In the exercises for this paradigm, sometimes the inverse of a learned pattern is retrieved. This is correct behavior. Why is this? Think of the exact form of the simple Hebbian learning rule.
These exercises repeat some of the exercises above but now with a competely different type of neural network paradigm.
Note that you may want to set to by setting the Cycle Time to 1000 in the
Settings panel if you want to just see the end result rather than the process.
5.a (5 pts) Create a small number of patterns or use the Face-House or Digit patterns. Learn each one in turn and check whether it can be retrieved. You can do that by activating it and then strongly distorting it by press. Can the network find the original pattern again?
5.b (2 pts) Reload the page to clear the weights. Learn all patterns in the Digits set. Try to retrieve each one in turn by simply activating it and pressing . What do you observe? Even though a Hopfield network should be able to store 0.15n patterns, with n = 100 here (number of nodes), this does not seem to hold here. What not? Can you think of a way to still make this work? (You don't actually have to execute it, but of course you may.)
5.c (2 pts) Reload the page. Learn just the two Face-House patterns. Note the energy for each pattern, which should be exactly equal. Now start with a random pattern by pressing . Then press . Which of the two patterns is retrieved? (Also count the inverted pattern as correct.) Repeat this say 10 times and write down whether Face or House was retrieved. What is the score?
5.d (2 pts) Keep the weights from 5.c. Activate House and learn it one extra time. House now has received two learning trials and Face one. Again, write down the energy for both. Repeat the random search of 5.c ten times. What do you observe and why do you think this is the case?
5.e (2 pts) In 5.c and 5.d you sometimes observe the inverted pattern on retrieval from a random pattern. What is this? Why does this rarely occur if you merely distort the pattern as in 5.a?
6.a (5 pts) Create and learn a couple of patterns. Verify that they can all be retrieved well. Then, lesion the network in steps and check whether the network can still retrieve the patterns in between lesions. How much do you have to lesion the network in order to still be able to retrieve it perfectly? You can select Connection View to see how much damage you have done in the end.
6.b (2 pts) Think back to the same exercise with the Willshaw network. Did you notice that you can lesion much more before you see any negative effects on retrieval? Why do you think this is the case?
This version of the Hopfield network uses continuous activations between 0 and 1 and a simple Hebbian learning rule. The activation rule is the so called logistic function
act = 1 / (1 + e-neti). This paradigm is included here for comparision. You may try one of the exercises above with this model. Do you notice any differences?