AI to understand and support people with disabilities Diversity & Inclusion Research 02

This series introduces researchers at the University of Tokyo studying diversity and inclusion from a variety of perspectives.

Project Professor Yukie Nagai, International Research Center for Neurointelligence

―― Your specialty is robotics, but you also study people with developmental disorders. Could you tell us how your research has evolved?

My initial motivation was a genuine desire to create a smart robot. However, I realized that even the most advanced artificial intelligence (AI) is not as intelligent as a 3-year-old child. Some recent AI technologies have surpassed human capabilities, but they still cannot perform very basic tasks, such as communicating with others and using words to speak. Eventually, my interest shifted to the question of how people acquire intelligence.

Around 1999, a new research field called cognitive developmental robotics was launched, led by Professor Minoru Asada at Osaka University and Professor Yasuo Kuniyoshi at the University of Tokyo. Researchers in this field aim to verify hypotheses drawn from neuroscience research by creating robots that learn and develop like humans. The researchers then use the findings from these studies to deepen their understanding of human intelligence. As I found the field so fascinating, I went to Osaka University for my doctoral studies.

Then around 2012, I met current Associate Professor Shinichiro Kumagaya at the Research Center for Advanced Science and Technology, and Satsuki Ayaya, currently a project lecturer there with autism spectrum disorder (ASD). They lead tojisha-kenkyu research, which is the study of oneself as someone living with a disability or illness. Through our joint research, I became interested in why learning and development don’t go smoothly for some people. Since then, while studying mechanisms of human development using AI, I have also tried to understand people with developmental disorders. They are said to have difficulties in communication, but I have wondered if they are really disabled, or if they have a different way of doing things.

―― Would you elaborate?

One project I’m working on is to create a robotic model that imitates the human brain using deep neural networks. Such networks are used in image processing and speech recognition.

We have made the model learn the imitation and drawing skills that children actually learn, and examined under what conditions that learning will be successful. It is not ethical to manipulate the human brain to disable certain functions. We are trying to understand how learning goes wrong by manipulating a neural network model and causing disruptions, such as blocking connections and adding “noise.”

Another research project is getting people with ASD and those around them to experience the sensory overload people with ASD encounter, by using head-mounted displays to reproduce their visual hypersensitivity. People with ASD are known to have difficulties in communicating or reading what other people are feeling, but first, many of them do not even know what is going on inside themselves or what their own emotional state is. Because they have difficulties in recognizing such things about themselves, they may not know how to deal with others. So it is important to make their condition “visible” to them.

There are several virtual reality devices and videos that have reproduced the experiences of people with schizophrenia and ASD, such as the hallucinations and visual hypersensitivity they encounter, but many of these simulations are based on personal accounts rather than being backed up by scientific findings. We collect data on the experiences of the individuals, then come up with hypotheses based on the best scientific knowledge we have to explore what function of the brain, eyes, ears and other sensory organs cause these experiences.

For example, the human brain spontaneously fires off electronic signals even when we are asleep. When it receives sound or other information from outside, the corresponding part of the brain gets activated; but even when our eyes are closed and ears are covered and no sensory information is coming in, the brain remains active — and that activity level varies from person to person. Research has shown that the brains of people with ASD seem to be overactive. As a result, they may visually perceive things differently, where things can appear as dots, or be blurry, or have lost color. I believe that by properly explaining each of these phenomena based on scientific evidence, we can expand our research, so that’s what I have been conducting research on together with the Kumagaya team.

―― I understand that symptoms vary vastly among people with ASD.

Yes. In fact, less than half of the people diagnosed with ASD experience visual hypersensitivity. On the other hand, auditory and tactile hypersensitivity have been reported to be very common. In the human brain, the roles of the visual cortex and auditory cortex are somewhat divided, but basically the same signal processing takes place in all areas of the brain. For example, we have asked people to watch a video of a busy intersection in the Shibuya district in central Tokyo and recount what kind of experiences they had in the past. The results show that people with ASD are more likely to experience auditory and visual hypersensitivity when they go to places that are noisy or surrounded by a lot of objects. In such settings, they are likely to see the dot pixel pattern of white noise or hear the static common to analog TVs. So we can hypothesize that certain environmental factors can trigger the spontaneous firing of electrical signals in the brain.

Until now, experts have tried to support or treat people with developmental disorders by attributing the problems to the individuals and trying to somehow change them. But we define disability as something that arises through interacting with the environment. We believe that by properly examining the environmental factors and changing the environment, we can resolve their difficulties and suffering. I think our approach will also help design an environment in line with “reasonable accommodation” stipulated by law.

―― In 2013, the diagnosis of autism spectrum disorder was introduced, and several categories of autism were combined and became part of the broader term. Will your research eventually show that there is a gradation in illnesses, or a spectrum, and that it is all part of people’s individuality?

That’s exactly what we have tried to show. Until now, typically developed people have been placed on one end of the spectrum, and people with developmental disorders or autism on the other. We believe that people who have been labeled as having developmental disorders are in fact at both ends of the spectrum, with those with typical development in the middle.

Through experiments using a neural network model, we have found that, by altering a certain parameter, you can turn people’s condition into two extremes. If you give a robot a moderate-level parameter, it learns well, but if you weaken or strengthen the level of the parameter, the robot develops different kinds of difficulties. Therefore, we believe that ASD is caused by factors at opposite extremes that result in different conditions, but such conditions have all been bundled into one disorder.

Also, we don’t need to set boundaries. For example, if you collect the data of 100 people, they are not clearly divided into two groups. Researchers are arbitrarily setting a boundary among people. I think that since there is no clear division, we should simply look at the individual differences as a spectrum.

―― What kind of society do you see yourself carving out?

I have given over 60 lectures at companies and schools, including at a medical reformatory for juveniles. Medical reformatories are home to many children with developmental disorders. Because of their disabilities, they have trouble blending into society and end up committing crimes, or they are unable to judge whether something is wrong or not. For those children and correction officials, we explain the concept of developmental disorders and have them simulate the difficulties through head-mounted displays.

Some parents of children with developmental disorders have broken into tears, and said they felt relieved. They are happy that, through the simulations, they came to understand their children, even partially. I feel I can contribute to society by providing an opportunity to understand the development and disorders based on scientific knowledge, even though a lot of it is still hypothetical.

Interview date: August 13, 2021
Interview/text: Tomoko Otake
Photos: Rohan Mehra