GabyReading a person’s thoughts doesn’t feel like science fiction anymore. Previously, scientists predicted a person’s words during a normal conversation by decoding electrical activity in the temporal lobe. Eleven years later, the same California team reconstructed a well-known Pink Floyd song from the brain waves of 29 people using nonlinear models.
As one of the study’s authors, Ludovic Belye, points out, non-invasive methods still aren’t accurate enough. He hopes that in the future it will be possible to record activity from deeper brain regions with good signal quality using electrodes placed externally on the skull.
How did they reconstruct a song from brain activity?
Scientists played a Pink Floyd song at a medical center in New York while neurobiologists prepared patients for surgery. Twenty-nine epilepsy patients voluntarily took part in the study; they had strips with a total of 2,668 electrodes surgically implanted in their brains.
Participants passively listened to a 1979 hit while being prepped for epilepsy surgery. They were told to listen carefully to the 190.72-second song without focusing on any particular detail.
In addition to reconstructing the stimulus, the researchers used a coding approach. That let them test whether models trained on recent speech data would generalize to music perception. Coding models predict the neural activity recorded at a single electrode based on a representation of the stimulus.
The team then used artificial intelligence to decode neural activity and successfully reconstructed the song from brain recordings. This was the first time a song was reconstructed from intracranial electroencephalography (iEEG) recordings.
One step closer to reading minds
The researchers reconstructed a recognizable song from direct neural recordings and quantitatively assessed how various factors affected decoding accuracy. They found a dominance of the right hemisphere in music perception, identified the superior temporal gyrus as a key region, and located a subregion of that gyrus tuned to musical rhythm.
The team believes this could be a genuine breakthrough for people with communication difficulties. Recordings from electrodes on the brain surface have been used before to decode speech. The current work may help recreate the musicality of speech, which could improve machine-based reconstructions.
In an interview with Interesting Engineering, co-author Professor Robert Knight described the result as impressive. He said that one feature of music is its prosodic and emotional content.
Knight added that as the field of brain–machine interfaces progresses, it may become possible to add musicality to future brain implants. That could help people with neurological or developmental disorders that impair speech, as well as those who have had a stroke or suffer paralysis. Researchers also hope that one day neural activity can be recorded noninvasively, using sensitive electrodes attached to the scalp to read thoughts.
