Just like a city, the brain’s overall operation emerges from the networked interaction of its innumerable parts. There is often a temptation to assign a function to each region of the brain, in the form of “this part does that”. But despite a long history of attempts, brain function cannot be understood as the sum of activity in a collection of well-defined modules.
Instead, think of the brain as a city. If you were to look out over a city and ask “where is the economy located?” you’d see there’s no good answer to the question. Instead, the economy emerges from the interaction of all the elements – from the stores and the banks to the merchants and the customers.
And so it is with the brain’s operation: it doesn’t happen in one spot. Just as in a city, no neighborhood of the brain operates in isolation. In brains and in cities, everything emerges from the interaction between residents, at all scales, locally and distantly. Just as trains bring materials and textiles into a city, which become processed into the economy, so the raw electrochemical signals from sensory organs are transported along super-highways of neurons. There the signals undergo processing and transformation into our conscious reality.
What would it be like to be locked in here for hours, or for days? To find out, I spoke to a surviving inmate who had been here. Armed robber Robert Luke – known as Cold Blue Luke – was sent to the Hole for twenty-nine days for smashing up his cell. Luke described his experience: “The dark Hole was a bad place. Some guys couldn’t take that. I mean, they were in there and in a couple of days they were banging their head on the wall. You didn’t know how you would act when you got in there. You didn’t want to find out.”
Completely isolated from the outside world, with no sound and no light, Luke’s eyes and ears were completely starved of input. But his mind didn’t abandon the notion of an outside world. It just continued to make one up. Luke describes the experience: “I remember going on these trips. One I used to remember was flying a kite. It got pretty real. But they were all in my head.” Luke’s brain continued to see.
Such experiences are common among prisoners in solitary confinement. Another resident of the Hole described seeing a spot of light in his mind’s eye; he would expand that spot into a television screen and watch TV. Deprived of new sensory information, prisoners said they went beyond daydreaming: instead, they spoke of experiences that seemed completely real. They didn’t just imagine pictures, they saw.
This testimony illuminates the relationship between the outside world and what we take to be reality. How can we understand what was going on with Luke? In the traditional model of vision, perception results from a procession of data that begins from the eyes and ends with some mysterious end point in the brain. But despite the simplicity of that assembly-line model of vision, it’s incorrect.
In fact, the brain generates its own reality, even before it receives information coming in from the eyes and the other senses. This is known as the internal model.
The basis of the internal model can be seen in the brain’s anatomy. The thalamus sits between the eyes at the front of the head and the visual cortex at the back of the head. Most sensory information connects through here on its way to the appropriate region of the cortex. Visual information goes to the visual cortex, so there are a huge number of connections going from the thalamus into the visual cortex. But here’s the surprise: there are ten times as many going in the opposite direction.
Visual information travels from the eyes to the lateral geniculate nucleus to the primary visual cortex (gold). Strangely, ten times as many connections feed information back in the other direction (purple).
Detailed expectations about the world – in other words, what the brain “guesses” will be out there – are being transmitted by the visual cortex to the thalamus. The thalamus then compares what’s coming in from the eyes. If that matches the expectations (“when I turn my head I should see a chair there”), then very little activity goes back to the visual system. The thalamus simply reports on differences between what the eyes are reporting, and what the brain’s internal model has predicted. In other words, what gets sent back to the visual cortex is what fell short in the expectation (also known as the “error”): the part that wasn’t predicted away.
So at any moment, what we experience as seeing relies less on the light streaming into our eyes, and more on what’s already inside our heads.
And that’s why Cold Blue Luke sat in a pitch-black cell having rich visual experiences. Locked in the Hole, his senses were providing his brain with no new input, so his internal model was able to run free, and he experienced vivid sights and sounds. Even when brains are unanchored from external data, they continue to generate their own imagery. Remove the world and the show still goes on.
You don’t have to be locked up in the Hole to experience the internal model. Many people find great pleasure in sensory deprivation chambers – dark pods in which they float in salty water. By removing the anchor of the external world, they let the internal world fly free.
And of course you don’t have to go far to find your own sensory deprivation chamber. Every night when you go to sleep you have full, rich, visual experiences. Your eyes are closed, but you enjoy the lavish and colorful world of your dreams, believing the reality of every bit of it.
Seeing our expectations
When you walk down a city street, you seem to automatically know what things are without having to work out the details. Your brain makes assumptions about what you’re seeing based on your internal model, built up from years of experience of walking other city streets. Every experience you’ve had contributes to the internal model in your brain.
Instead of using your senses to constantly rebuild your reality from scratch every moment, you’re comparing sensory information with a model that the brain has already constructed: updating it, refining it, correcting it. Your brain is so expert at this task that you’re normally unaware of it. But sometimes, under certain conditions, you can see the process at work.
Try taking a plastic mask of a face, the type you wear on Halloween. Now rotate around so you’re looking at the hollow backside. You know it’s hollow. But despite this knowledge, you often can’t help but see the face as though it’s coming out at you. What you experience is not the raw data hitting your eyes, but instead your internal model – a model which has been trained on a lifetime of faces that stick out. The hollow mask illusion reveals the strength of your expectations in what you see. (Here’s an easy way to demonstrate the hollow mask illusion to yourself: stick your face into fresh snow and take a photo of the impression. The resulting picture looks to your brain like a 3D snow sculpture that’s sticking out.)
When you’re confronted with the hollow side of a mask (right), it still looks like it’s coming towards you. What we see is strongly influenced by our expectations.
It’s also your internal model that allows the world out there to remain stable – even when you’re moving. Imagine you were to see a cityscape that you really wanted to remember. So you take out your cell phone to capture a video. But instead of smoothly panning your camera across the scene, you decide to move it around exactly as your eyes move around. Although you’re not generally aware of it, your eyes jump around about four times a second, in jerky movements called saccades. If you were to film this way, it wouldn’t take you long to discover that this is no way to take a video: when you play it back, you’d find that your rapidly lurching video is nauseating to watch.
So why does the world appear stable to you when you’re looking at it? Why doesn’t it appear as jerky and nauseating as the poorly filmed video? Here’s why: your internal model operates under the assumption that the world outside is stable. Your eyes are not