Neuroscientists are teaching computers to read words straight out of People's Brain .
Kelly Servick, writing for Science, this week on three papers posted to the preprint server bioRxiv in which three different teams of researchers demonstrated that they could decode speech from recordings of neurons firing. In each study, electrodes placed directly on the brain recorded neural activity while brain-surgery patients listened to speech or read words out loud. Then, researchers tried to figure out what the patients were hearing or saying. In each case, researchers were able to convert the brain's electrical activity into at least somewhat-intelligible sound files.
The First Paper, posted to bioRxiv on Oct. 10, 2018, describes an experiment in which researchers played recordings of speech to patients with Epilepsy who were in the middle of brain surgery. (The neural recordings taken in the experiment had to be very detailed to be interpreted. And that level of detail is available only during the rare circumstances when a brain is exposed to the air and electrodes are placed on it directly, such as in brain surgery.)
As the patients listened to the sound files, the researchers recorded neurons firing in the parts of the patients' brains that process sound. The scientists tried a number of different methods for turning that neuronal firing data into speech and found that " deep learning " — in which a computer tries to solve a problem more or less unsupervised — worked best. When they played the results through a vocoder, which synthesizes human voices, for a group of 11 listeners, those individuals were able to correctly interpret the words 75 percent of the time.
Comments
Post a Comment