We now have the ability to hear another person’s thoughts. Researchers have identified the differences in brain activity linked to heard and imagined sounds, a finding that could lead to better communication devices for people who are fully paralysed.
In 2014, at the University of California, Berkeley, and his colleagues eavesdropped on a person’s internal monologue for the first time. They got several people who had electrodes implanted in their brain to read text out loud while having their brain activity recorded. The team used this data to work out which neurons reacted to particular aspects of speech, and created an algorithm to interpret this information. They were then able to analyse the brain activity of people who were imagining speaking, and ch.
But their algorithm wasn’t perfect – it could only translate brain activity into extremely crude aspects of speech, which weren’t often easy to understand. To get clearer translations, they needed a better understanding of how the brain activity responsible for imagined sound differs from activity associated with actually hearing a real sound.
Advertisement
Imagine that tune
Distinguishing between these two types of brain activity is a challenge, because it’s difficult to know exactly when someone is imagining a specific word and measure the activity associated with this.
To get around this problem, Pasley and his colleagues performed an experiment on a pianist who had electrodes already implanted in his temporal cortex to monitor his epilepsy. This region of the brain is responsible for hearing.
The team used these electrodes to record the pianist’s brain activity while he played Bach or Chopin pieces on an electric piano. By also recording when he pressed each piano key, the researchers were able to match the sound of the note with the exact brain activity that processed it.
The team then asked the pianist to play the pieces again, but with the sound off. This time, he was instructed to imagine the sound as he played the tunes. This allowed the researchers to again match each finger press with brain activity – but this time the brain activity corresponded with imagined sound instead of heard sound.
Same but different
Some groups of neurons were very active during imagining a sound, but not when actually hearing a sound, and vice versa, says Pasley. Many neurons acted in exactly the same way whether the sound was real or imagined. “Some neurons care about a specific frequency when you’re listening to a real sound. The same set of neurons care about that same frequency when you’re imagining that sound,” he says.
The team added this information into an algorithm that predicted what sounds the pianist was thinking about based on his brain activity. This extra data made the algorithm’s predictions 50 per cent more accurate.
The team now hope to build the new algorithm into a device that can more accurately predict speech from thought alone. “The ultimate goal is to produce a speech prosthetic device that can be used by people who are severely paralysed or locked-in,” says Pasley.
Cerebral Cortex
Read more: Mind-reading AI uses brain scans to guess what you’re looking at; Mind reading typing tool for paralysed people is fastest yet
