On Hawkins, Hierarchy, Homunculi and the Hippocampus

On Intelligence

As previously noted, Jeff Hawkins takes a very similar approach to Karl Friston’s ‘variational Free Energy’ with the concept of ‘hierarchical message passing’ playing a key role in both. In such hierarchical message passing, it is clear how levels in the middle of the hierarchy and at the bottom perform:

• Those in the middle of the hierarchy interact with layers both above and below.
• The layer at the bottom of the hierarchy interacts with the outside world.

But, what happens at the top? The highest level has nowhere to feed prediction errors to or receive predictions from. This is what a previous blog entry  asked.  A top level would be the ultimate decider of our actions, with nothing to defer to – a single centre of control. It is like the last ‘little man inside our heads, in the Homunculus Argument . It suggests that there is a single small region in the brain that is the coherent seat of the what Daniel Dennett derisively calls the ‘Cartesian Theatre’.

In order to avoid falling into this pit, I postulated that there was in fact no top; that this was just an artifact of the gross simplification in presenting a simple hierarchy, with each level communicating to one and only one region in the level above and precisely one region in the level below. But if they can communicates upwards and downwards with more than one region (as suggested in Friston’s diagram, right) it becomes possible for the hierarchy to loop back on itself, with regions feeding upwards to levels that were also below below it, and vice versa. The clean hierarchy therefore breaks down, but the basic concept of the operation of each block dealing with predictions and prediction errors remains intact.

Yet Felleman and Van Essen’s famous diagram of the visual processing of a Macaque monkey (right) puts the hippocampus at the top (marked ‘HC’). At the bottom of the hierarchy, signals from the Retinal Ganglion Cells (‘RGC’) within the eye pass through the Lateral Geniculate Nucleus (‘LGN’) in the Thalamus to the primary visual region, ‘V1’, and thence up to the Entorhinal Cortex (‘ER’) and finally the hippocampus.

And in ‘On Intelligence’ , ‘On Intelligence’ (2005) Hawkins explicitly maintains that the hippocampus is at the top of the hierarchy and provides a very nice account of learning and  relearning across levels of the hierarchy. (The bold emphasis is mine)…

“When you are born your cortex essentially doesn’t know anything.…you need outside instruction to help you decide which patterns belong together. …your brain slowly builds sequences of patterns that belong together.” (page 165)

“The unexpected result of the learning process is that, during repetitive learning, representations of objects move down the cortical hierarchy. During the early years of your life, your memories of the world first form in higher regions of cortex, but as you learn they are re-formed in lower and lower parts of the cortical hierarchy … It isn’t that the brain moves them; it has to relearn them over and over. (I am not suggesting that all memories start at the top. … It is the memory of sequences I am suggesting re-form lower and lower in the cortex …). … Consider how we learn to read. The first thing we learn is to recognize individual printed letters. This is a slow and difficult task requiring conscious effort. Then we move onto recognizing simple words … After years of practice, a person can read quickly. It isn’t just that we are faster; we are actually recognizing words and phrases as entities. … A young brain is slower to recognize inputs … because the memories used in these tasks are higher up the cortical hierarchy. Information has to flow all the way up and down, maybe with multiple passes, to resolve conflicts.” (pages 166-167)

“Experts and geniuses have brains that see structure of structure and patterns of patterns beyond what others do. You can become expert by practice.” (page 168)

“Under the neocortical sheet: basal ganglia, cerebellum and hippocampus. All three existed prior to the neocortex. …To some extent the neocortex has subsumed their original functions. For example, a human born without much of a cerebellum will have deficits in timing … but otherwise will be pretty normal. The hippocampus, however, is a different beast … it is essential for the formation of new memories. If you lose both halves of the Hippocampus … you lose the ability to form most new memories.” (pages 168-169)

“For many years …it didn’t make sense to me. The classic view of the hippocampus is that new memories are formed there, and later, over a period of days, weeks or months, these new memories are transferred to the neocortex.” (page 169)

“…the connections between the hippocampus and the neocortex suggest that the hippocampus is the top region of the neocortex. ….If a region doesn’t understand the current input, it passes it up the hierarchy until some higher region does. … when you get to the top of the cortical pyramid, what you have left is information that can’t be understood by previous experience.” (pages 170-171)

“It is these unexplained remainders, the new stuff, that enter the hippocampus and are stored there. This information won’t be stored forever. Either it will be transferred down into the cortex below or it will eventually be lost.” (page 171)

“I have seen so many plays in my life that I rarely do I see anything truly new. New plays fit into memories of past plays, and the information just doesn’t make it to the hippocampus. For my children, each play is more novel and does reach the hippocampus. … the more you know the less you remember. (page 171)

The Hippocampus