The Challenge
Imagine waking up one day and not being able to move any part of your body. You try to call for help but discover you cannot speak. You attempt to look around the room to see if anyone is nearby, but your eyes won’t move either. Your mind is working perfectly, but you are trapped in your own body.
Project Details
People with this rare horrific condition, known as “locked-in syndrome”, can become this way almost instantly from a brainstem stroke or severe trauma, or more either gradually from amyotrophic lateral sclerosis (ALS, otherwise known as Lou Gerhig’s Disease). In some cases, people will continue to maintain control over the eyes, allowing them to use assistive technologies like eye-tracking devices to communicate through a computer. Unfortunately, for those who lose eye control, communication is significantly more challenging and solutions are severely limited.
Our Approach
Area 10 Labs developed brain-machine interfaces (BMI, also known as brain-computer interfaces or BCI) that enables people to interact with and control machines through our custom-designed functional brain imaging system. These systems, which use functional near-infrared imaging (fNIR), monitor the brain’s activity in real-time to detect what types of mental tasks a person is doing. By giving the subject a choice of a few tasks to select from, the person is able to create a signal that can be interpreted by our software and thereby allow them to manipulate basic computer interfaces.
In our research lab testing our system on healthy people, everything appeared to function perfectly. The real test was when we visited a man who, because he was locked-in, had not been able to communicate with his family in years. The Area 10 Labs team looked on anxiously as the sensors were placed on his head and the computer started receiving data. As with many studies involving human subjects, our first tests did not work. But, over the course of several days, we worked through a number of challenges and were able to help this man answer several yes or no questions that his family wanted to ask him.
For the first time in years, he communicated!
After our initial success, we continued to improve our software for communication to improve its reliability and the speed with which someone could communicate. In parallel, we also worked on some fun applications to give locked-in people a way to entertain themselves while practicing the required mental tasks that allow them to control the system for communications. The applications included video games and painting applications. The painting application is discussed more below.
Brain Painting
Area 10 Labs has developed a BCI called “brain painting”. This application allows someone to paint through consciously modifying the level of activity in a region of his or her brain. Typically this means either “singing in your head” or repeating nonsense syllables in your head (such as “la la la”). The first activity activates the language area, thereby raising the signal measured by OTIS, whereas the second activity lowers the signal. In addition to being a fun creative tool, brain painting also helps people learn the skills necessary to use a BCI effectively for communication.
For Brain Painting, Area 10 Labs worked closely with the late artist, Peggy Chun, whose tropical watercolor paintings made her a household name in Hawaii. Peggy was diagnosed with ALS in 2002, but never let the disease stop her from painting. As she became paralyzed, she switched to painting with her left hand, then by holding the paintbrush in her teeth. Even when she was only able to move her eyes, Peggy used an eye-tracking system to communicate and paint. At Area 10 Labs, we helped Peggy become the world’s first ‘brain painter’. Sadly, Peggy passed away in 2008, but her memory and spirit live on in her beautiful paintings.
