Subjects viewed a random sequence of images of faces and houses and were asked
to look for an inverted house like the one at bottom left. "That was a distractor,"
Jeff Ojemann said. "We were interested in what the brain was doing at the other times."
Illustration by Kai Miller and Brian Donohue
(January 28, 2016) Electrodes in patients’ temporal lobes carry information that, when analyzed, enables scientists to predict what object patients are seeing
Using electrodes implanted in the temporal lobes of awake patients, scientists have decoded brain signals at nearly the speed of perception. Further, analysis of patients’ neural responses to two categories of visual stimuli – images of faces and houses – enabled the scientists to subsequently predict which images the patients were viewing, and when, with better than 95 percent accuracy.
The research is published today in PLOS Computational Biology.
University of Washington computational neuroscientist Rajesh Rao and UW Medicine neurosurgeon Jeff Ojemann, working their student Kai Miller and with colleagues in Southern California and New York, conducted the study.
“We were trying to understand, first, how the human brain perceives objects in the temporal lobe, and second, how one could use a computer to extract and predict what someone is seeing in real time?” explained Rao. He is a UW professor of computer science and engineering, and he directs the National Science Foundation’s Center for Sensorimotor Engineering, headquartered at UW.
The numbers 1-4 denote electrode placement in temporal lobe,
and neural responses of two signal types being measured.
“Clinically, you could think of our result as a proof of concept toward building a communication mechanism for patients who are paralyzed or have had a stroke and are completely locked-in,” he said.
The study involved seven epilepsy patients receiving care at Harborview Medical Center in Seattle. Each was experiencing epileptic seizures not relieved by medication, Ojemann said, so each had undergone surgery in which their brains’ temporal lobes were implanted – temporarily, for about a week – with electrodes to try to locate the seizures’ focal points.
Neuroscientist Rajesh Rao and neurosurgeon Jeff Ojemann
are faculty at the University of Washington.
“They were going to get the electrodes no matter what; we were just giving them additional tasks to do during their hospital stay while they are otherwise just waiting around,” Ojemann said.
Temporal lobes process sensory input and are a common site of epileptic seizures. Situated behind mammals’ eyes and ears, the lobes are also involved in Alzheimer’s and dementias and appear somewhat more vulnerable than other brain structures to head traumas, he said.
In the experiment, the electrodes from multiple temporal-lobe locations were connected to powerful computational software that extracted two characteristic properties of the brain signal: “event-related potentials” and “broadband spectral changes.”