Using an eye tracker in autism research – autism and the social mind

Social learning relies on a natural inclination to watch people’s faces and eyes. Dr Giacomo Vivanti explains how the technology of eye tracking can afford a new window into the mental world of individuals with autism.

A key question for science to explore in the twenty-first century concerns the mechanisms that underlie social behaviour. How do we understand other people’s thoughts, beliefs and intentions? To what extent can our mental states and feelings be shared with other people? And how do we incorporate other people’s thoughts, feelings and beliefs into our mental world?

We are programmed to navigate the social world. We are very good at reading other people’s minds—a quick look at another person is enough to allow us to read behind the surface and capture their underlying feelings. We like the feeling of being interconnected—the success of new social technologies such as Facebook or Twitter reflects this inclination.

And we fear a lack of human contact—for example, solitary confinement affects a prisoner’s mental health more severely than any other legal form of punishment. When we are not actively interacting with other people, we think about other people—and we think about what they might think about us—so that our elaborate plans, our primitive feelings and everything in between are influenced by the watchful eyes of others.

When do we start to be obsessed with those eyes? When do we open our private consciousness to the cumbersome presence of others’ mental states? In “The Gay Science,” Friedrich Nietzsche wrote that human consciousness “…is really only a net of communication between human beings (….) Consciousness does not belong to man’s individual existence but rather to his social nature.”

So there is first a conscious “We,” and then an illusory “I.” In the Twenties, Sigmund Freud introduced the idea that our mind does not host the real thoughts or feelings of other people, but rather internalizes perceived (and often distorted) desires, thoughts and judgments of others. This is the so-called superego, an internalized set of others’ expectations that shapes people’s development and provides a continuous negotiation (or conflict) between one’s own desires, goals and beliefs and those of others. While the relevance of these conscious and unconscious dynamics is reflected in art, culture, rituals and philosophical and scientific works across ages and cultures, little is known about the underlying mechanisms.

How do we get other people’s real or perceived thoughts to find a place in our private cognition? Recently, this issue has gained significant attention among scholars in the new discipline of developmental cognitive neuroscience. In particular, researchers interested in processes of social development and inter-subjectivity have demonstrated over the last ten years a special interest in autism.

Compared to typically developing peers, individuals with autism are less inclined to engage spontaneously in activities such as observing and imitating other people, initiating social and communicative interactions and responding to social stimuli. There is a “spectrum” of effects from severe to very mild, so that the clinical presentation of autism is extremely varied. As the writer Nick Hornby (whose son has autism) wrote: “Autism (…) can be anything from not being able to recognize your own parents to being a bit funny when you’re studying math’s at Oxford.”

The one thing common to all individuals with autism is that they seem not to have much talent in understanding, predicting and, in a way, “internalising” what other people do. As a clinician and as a researcher, I have spent the last ten years working on capturing the nature of these difficulties.

Recently I had an interesting conversation with one of my patients. Ludwig is a 12-year-old child with autism. He is very talkative and very smart. He was pretty cooperative that day, so I suggested that we play a new game about animals (he loves animals). He accepted.

“These are the rules,” I said. “I’m going to think about an animal, and then you have to guess what animal I am thinking about. You can ask me all kinds of questions. I’ll give you a clue: It starts with F.”

Ludwig asked me immediately: “Which animal is it? A cat?”

“Well, you can’t ask me directly which animal it is,” I said. “You have to guess. You can ask me if it’s big, or small, or—”

“Is it big?”

“Not very big. It’s the size of my hand.”

“Is it small?”

“The size of my hand.”

“Maybe… a cat?”

“No. It starts with F and it is as small as my hand.”

“A small cat.”

“It’s not a cat. I’ll give you another clue: It’s an animal that lives near the water, and it jumps.”

“Maybe a cat that lives near the water, or a cat that jumps…”

“No,” I said. “It’s green and it starts with F.”

“A cat that—”

“It’s a frog.”

“A frog,” Ludwig said, looking surprised.

“Now it’s your turn,” I said. “Think about an animal, and I have to guess the animal you are thinking about.”

“Okay, I’m going to think about an animal,” he said, and then, “It starts with C.”

“Maybe a cat?” I said.

“Yes!”

What’s interesting is that this child has an IQ above the average. Why was this simple game so difficult for him? Why couldn’t he “see” my frog as clearly as I could see his cat? Was it that he couldn’t generate the image of the frog? Or maybe the problem was getting rid of the cat? Why was it that the frog that was in my mind could not be hosted in his mind? These difficulties that people with autism have with “seeing” the mental world of other people inspired a series of experiments that I have conducted for the last few years employing a new technology called eye tracking.

Eye tracking is the process of measuring the focus of attention, or where a person is looking. Recent technologies allow for a very fine-grained analysis of this process. The eye tracker uses contrast to locate the center of the pupil and use infrared light to create a corneal reflection: the vector between these two features is then used to compute gaze intersection with a surface. If you are watching for example, Leonardo Da Vinci’s Mona Lisa, the eye-tracker can tell me exactly what you are looking at (Mona Lisa’s eyes? Her smile? The tree in the background?).

The beauty of eye tracking is that it allows us to measure a person’s attentional focus without the need for them to know or even to understand what they’re doing. When we are watching the Mona Lisa painting at the Louvre Museum we are not aware of what exactly we are looking at (and the temporal pattern of our visual attention: are we watching the face first and then the body? The eyes first and then the mouth?), unless we are specifically instructed to pay attention to a particular detail. This technology therefore affords a new window into the mental world of other people, including people with neurodevelopmental disorders whose minds have been always considered “impenetrable.” It’s the closest we can get to seeing the world through an autistic child’s eyes.

We employed the eye-tracking technology to examine what children with autism and typically developing children look at when actions are being demonstrated to them. While wearing special eye-tracking headgear, participants in the study (20 children with autism, and 20 with typical development) were asked to observe video clips showing a person demonstrating an action. After viewing each clip, the children performed the demonstrated action. The results confirmed previous research that shows that children with autism have difficulties imitating people when compared to normally developing children. The most surprising finding of the study, however, came from the eye-tracking analysis of the focus of visual attention.

Our results showed that children with autism and those without autism looked at the demonstrator’s actions for similar durations; however typically developing children looked to the demonstrator’s face twice as long as the group of children with autism did. The analysis of the visual attention patterns showed that children without autism were constantly shifting their attention between the demonstrator’s face and his actions, while children with autism were focusing on the action, with little or no attention to the demonstrator’s face. In other words: children with autism were imitating an action, while typically developing children were imitating a person performing an action.

In another experiment, we asked participants with and without autism to predict the demonstrator’s actions (after watching videos in which the actor starts but doesn’t finish an action), rather than just imitate them. We found again that children with autism were focusing on the demonstrator’s actions but not on his face (as opposed to what was observed in typically developing children)–as a consequence they would fail to understand the demonstrator’s intentions.

What does this tell us about autism and the social mind? Social learning (gaining knowledge by observing others) relies on a natural inclination to watch people’s faces and eyes. It’s more than an inclination, it’s a cognitive bias. And this is how we incorporate others’ knowledge into our repertoire: we know exactly where to look at and how to read behind the surface of a gaze and facial expression. Our research suggests that, when a child with autism and a typically developing child are observing someone’s action, they are both observing the action, but they are not encoding the same information. They have difficulties seeing what’s behind words and actions–the why beyond the what.

As a species, we are the world experts in social learning and social understanding because we practice these skills continuously: from infancy on we look at people’s eyes and facial expressions, we learn from what they say or do, and we develop this tremendous expertise that allows us to see the mental states underlying actions and words. Ludwig couldn’t see my frog because he is not an expert in social learning and social understanding. Our conversation on animals was not a shared space where we could see each other’s animals (my frog and his cat), but rather a space where his knowledge on animals couldn’t host information coming from another mind. Whether Ludwig feels like he is missing something by not seeing my frog, is a different question. He looked quite happy with his cat.

This blog first appeared in Another Piece, News of the Olga Tennison Autism Research Centre.