A Picture of the Brain from Early Stages of Visual Processing


In his book The Age of Insight: The Quest to Understand the Unconscious in Art, Mind, and Brain from Vienna 1900 to the Present, the neuroscientist Eric Kandel uses the work of the fin de sec Viennese artists Gustav Klimt, Oskar Kokoschka, and Egon Schiele to explore what happens in our brains and minds when we behold great art. In the process, he explains how both visual and emotional perception contribution to the appreciation and creation of art. In this post, I will draw on Kandel’s explication of visual perception to look at the increasing levels of abstraction that underlie higher-order thinking, and I will attempt to apply this to our comprehension of how the brain works.

Visual perceptions are not mental photographs of the environment—the brain instead uses multiple analytic steps to construct images from the stream of photons that enter our eyes. The process begins in the retina itself. The retina's cone cells, which are sensitive to detail and color, are concentrated in the center, or fovea, of the retina, and are less frequent in the periphery. A much larger number of rod cells, which are more sensitive to light, is distributed in the periphery. These are inactive in daylight or even normal indoor light but are responsible for night vision. Insensitive to detail, the rods use a more holistic analysis to identify aspects of a scene that may be missed by the cones. The cones see the trees, as it were, and the rods, the forest.

Further, the rods feed primarily into a rapid “where” pathway, which is sensitive to location and motion. This rapid pathway also contributes to directing gaze. Information from the cones, on the other hand, travels by a slower “what” pathway that identifies people, objects and scenes. Both pathways run from the retina through the lateral geniculate nucleus of the thalamus to the primary visual cortex at the bottom edge of the occipital lobe. From there the what pathway travels via several more steps in the occipital cortex to the inferior temporal cortex. The where pathway extends from the primary visual cortex directly to the posterior parietal cortex.

This already raises a number of interesting questions. Information about some aspects of a scene—location and motion—arrives in a different part of the brain at a different time than information about color, shape, and, as we shall see, identification of faces and other familiar objects. Even though the time difference is only milliseconds, this dynamic process has potential for confusion and ambiguity. Since direction of gaze relies on perception of larger-scale gestalts, it may be hard to pick up detail if gaze is distracted by motion elsewhere in the visual field. As an analogy, imagine such a system for the perception of touch, in which the information about location and motion of your lover’s or child’s hand was processed separately from its shape and texture and warmth. The two streams of information must come together to experience an integrated perception. This is the “binding problem” at the center of theories about consciousness. As Kandel explains, binding takes place not in a particular location but as a process involving attention and hypothesis-testing which draws on memories of prior perceptions.

To return to the retina, the cones in the fovea cannot perceive a whole face at once—it is too large for the fovea’s narrow field of view, so it scans the face, starting with the eyes, then the mouth, then other aspects of the face. At the same time, the rods generate a gestalt of the entire face. The cells of the primary visual cortex are specialized to respond to line segments with particular orientations. Thus they abstract information from the signals they receive from the retina rather than simply portraying it in image form.

A second level of processing takes place in the occipital regions of above the primary visual cortex. There line segments are assembled, with specific orientations. These are then connected in contours which define the boundaries of objects and distinguish them from their backgrounds. Kandel notes that artists have long recognized the power of lines and implied lines—even black-and-white cartoons and simple line drawings are able to convey detailed impressions far beyond what is in the pixels on the page. The brain is constructing meaning as it abstracts and assembles percepts.

This brings us to higher levels of visual processing, which takes place first between the higher visual cortical regions in the occipital lobe and the inferior temporal lobe, and later involves other cognitive areas of the brain, including the prefrontal cortex. Early on, faces are recognized, then expressions on faces, and then faces of particular people. Evolution has given face recognition has a special place in visual perception. We quickly identify an image as a face of a certain age, race, and gender; the owner of the face; his or her emotional state; and the direction of the person’s gaze. We also prioritize to a lesser extent recognition of hands and their gestures, other body parts, and bodies as wholes.

At the higher levels of processing in the inferior temporal lobe and the other cortical areas to which it is connected, we integrate visual perceptions with information from our memories, under the influence of several emotional and nonconscious survival systems.

A grossly simplified analogy would be a beehive swarming with activity—insects arriving, departing, dancing, depositing nectar, and feeding larvae in a complex of activities only to a trained entomologist can understand. The brain is unimaginably more complex and interconnected than a beehive, and it operates at a vastly higher speed.

What can we take from this? First, vision from its lowest levels is an active, analytic process, in which the brain constructs complex images from the information contained in photons reflected from objects in the environment. Second, the visual circuits are a hodgepodge of elements cobbled together by evolution and bear little resemblance to what an engineer might design from scratch. This makes for imprecision and ambiguity but also flexibility: the brain's redundant, interconnected design is likely to be better than a computer at evaluating something it has never seen before. Third, at the early levels, specialized areas of the nervous system perform identifiable transformations of the information; these feed into higher levels which conduct further transformations and analyses, but by the time the information gets to the inferior temporal lobe, complex circuits underlying conscious are involved, and how these integrate the information is not well understood.

Further reading: Eric Kandel, The Age of Insight: The Quest to Understand the Unconscious in Art, Mind, and Brain from Vienna 1900 to the Present, 2012.

Picture: Gustav Klimt, Judith and the Head of Holofernes

Recent Posts