Communication of New-Era : Thought Powered Communication
Call it “mindwriting.” The combination of mental effort and state-of-the-art technology have allowed a man with spinal injury and immobilized limbs, to communicate by text at speeds rivaling those achieved by his able-bodied peers texting on a smartphone. Scientists at Stanford University, Howard Hughes Medical Institute, and Brown University, developed an implanted brain-computer interface (BCI) technology that uses artificial intelligence to convert brain signals generated when someone visualizes the process of handwriting, into text on a computer, in real time.
A 65-year-old man had two grids of tiny electrodes implanted on the surface of his brain. The electrodes read electrical activity in the part of the brain that controls hand and finger movements. Although the man was paralyzed from the neck down, he imagined writing letters softly with his hand. With an algorithm, researchers then figured out the neural patterns that went with each imagined letter and transformed those patterns into text on a screen.
With the implants in the right place, the researchers asked the participant to imagine writing letters on a page and recorded the neural activity as he did so!
Altogether, there were roughly 200 electrodes in the participant’s premotor cortex. Not all of them were informative for letter-writing. But for those that were, the authors performed a principal component analysis, which identified the features of the neural recordings that differed the most when various letters were imagined. Despite working with a relatively small amount of data (only 242 sentences’ worth of characters), the system worked remarkably well. The lag between the thought and a character appearing on screen was about half a second, and the participant was able to produce about 90 characters per minute, easily topping the previous record for implant-driven typing, which was about 25 characters per minute.
The tests were all done with prepared sentences. Once the system was validated, however, the researchers asked the participant to type out free-form answers to questions. Here, the speed went down a bit (75 characters a minute) and errors went up to 2 percent after autocorrection, but the system still worked.
Krishna Shenoy, PhD, a Howard Hughes Medical Institute (HHMI) Investigator and the Hong Seh and Vivian W. M. Lim Professor at Stanford University, a study co-author of a highly acclaimed published study which is titled, “High-performance brain-to-text communication via handwriting, stated ,“Just think about how much of your day is spent on a computer or communicating with another person. Restoring the ability of people who have lost their independence to interact with computers and others is extremely important, and that is what is bringing projects like this one front and center.” The study is part of the BrainGate clinical trial, directed by Leigh Hochberg, MD, a neurologist and neuroscientist at Massachusetts General Hospital, Brown University, and the Providence VA Medical Center.
