Jeff Hawkins Q&A

October 13, 2005

Jeff Hawkins, the chief technology officer of Palm, was the founder of Palm Computing, where he invented the PalmPilot, and also the founder of HandSpring, where he invented the Treo. But Palm and creating mobile devices are only a part-time job for Hawkins. His true passion is neuroscience. Now, after many years of research and meditation, he has proposed an all-encompassing theory of the mammalian neocortex. "Hierarchical Temporal Memory" (HTM) claims to explain how our brains discover, infer, and predict patterns in the phenomenal world.

If Hawkins is right, he has succeeded where professional neuroscientists have failed. This year, he founded the company Numenta, which hopes to develop technology based on his theory. I talked to him at Technology Review's Emerging Technologies Conference at MIT.

Jason Pontin: How unconventional is this new model of yours, "Hierarchical Temporal Memory." If I asked a brain scientist, what would they say?

Jeff Hawkins: Some would say it's important stuff. There are very famous neuroscientists who read a book I wrote last year, called On Intelligence, and who wrote some complimentary things. But, to be candid, there are also people who don't feel that way. They don't say I don't know neuroscience. What they object to is the audacity of someone saying he can solve such a big problem as a global theory of the brain. They say: "It's not that simple." Many neuroscientists don't believe that the neocortex works on a common algorithm. They don't know what it works on, but they have a problem believing that vision is the same thing as hearing.

JP: It does seem wildly improbable. You are proposing that the neocortex is a "belief propagation network" -- a kind of machine that generates more or less accurate ideas about the world? How could such a thing evolve?

JH: It's not that difficult. Nothing in nature just springs into being. The neocortex evolved from structures that existed before. A reptile has a sophisticated brain. The neocortex added value to that brain. It allowed early mammals to see just a little bit into the future. The mammal could say, "I recognize this spot. I know there's food just around the corner." And it was so successful, so quickly, that the neocortex developed very fast. The brain just kept on adding circuits. But why is the neocortex a belief propagation network? I don't know! It just is.

JH: Yes. I think I understand what consciousness is now. There are two elements to consciousness. First, there is the element of consciousness where we can say, "I am here now." This is akin to a declarative memory where you can actively recall doing something. Riding a bike cannot be recalled by declarative memory, because I can't remember how I balanced on a bike. But if I ask, "Am I talking to Jason?" I can answer "Yes." So I like to propose a thought experiment: if I erase declarative memory, what happens to consciousness?" I think it vanishes.

But there is another element to consciousness: what philosophers and neuroscientists call "qualia:" the feeling of being alive. Qualia mean different things to different people, but the way I like to think about them is to ask, "Why does anything feel like anything?" And I think I understand this a little, too. Qualia have to do with the world itself: I perceive the world in a certain way because that's the way the world really is.

JP: Is a dolphin conscious?

JH: They've got a very highly developed neocortex. I bet they are. The only difference between you and me and dolphins is that they have a very limited motor cortex. They can reason; but they can't control motor behavior. Imagine! Their perceptual world is probably very rich. But they can't communicate those perceptions to each other. They have no real language, just songs from deep inside their reptile brain. It's like they have a robotic body. All they can do is fin through the sea.

JP: Why would Numenta want to build an HTM? After all, there are already billions and billions of human HTMs. We can make billions more from sexual intercourse.

JH: [Laughs.] Well, we wouldn't use artificial HTMs to do things that humans can already do. But an artificial HTM could do things that humans can't. We could use them to recognize patterns using exotic sensors. Maybe I could use weather sensors all around the world, and if I fed them into an HTM, it would perceive the weather like you and I perceive that building. HTMs could think in the higher mathematical dimensions or they could see how proteins fold. You could create an entire sensory world of things that humans just have problems seeing and predicting because we just didn't evolve in the right time-frames or scales. People say, "Jeff you shouldn't talk about stuff, because people will think you're crazy," but I say, I think this stuff is really going to happen.