Researchers at UC San Francisco have enabled a paralyzed man to control a robotic arm using a device that transmits signals from his brain to a computer.

By simply imagining himself performing the actions, he was able to grasp, move, and drop objects.

The device, known as a brain-computer interface (BCI), functioned for a record seven months without requiring adjustments. Until now, such devices had only worked for a day or two.

The BCI relies on an AI model that adapts to subtle changes in the brain as a person repeats a movement—or, in this case, an imagined movement—gradually refining their ability to perform it.

“This blending of learning between humans and AI is the next phase for these brain-computer interfaces,” said neurologist, Karunesh Ganguly, MD, PhD, a professor of neurology and a member of the UCSF Weill Institute for Neurosciences. “It’s what we need to achieve sophisticated, lifelike function.”

The study, which was funded by the National Institutes of Health, appears March 6 in Cell.

The key was the discovery of how activity shifts in the brain day to day as a study participant repeatedly imagined making specific movements. Once the AI was programmed to account for those shifts, it worked for months at a time.

Location, location, location

Ganguly studied how patterns of brain activity in animals represent specific movements and saw that these representations changed day-to-day as the animal learned. He suspected the same thing was happening in humans, and that was why their BCIs so quickly lost the ability to recognize these patterns.

Ganguly and neurology researcher Nikhilesh Natraj, PhD, worked with a study participant who had been paralyzed by a stroke years earlier. He could not speak or move.

He had tiny sensors implanted on the surface of his brain that could pick up brain activity when he imagined moving.

One of the study’s participants, who lost his ability to move and speak many years ago by a stroke, moves the robotic arm simply by imagining the movements in his head. Credit: UCSF

To see whether his brain patterns changed over time, Ganguly asked the participant to imagine moving different parts of his body, like his hands, feet, or head.

Although he couldn’t actually move, the participant’s brain could still produce the signals for a movement when he imagined himself doing it. The BCI recorded the brain’s representations of these movements through the sensors on his brain.

Ganguly’s team found that the shape of representations in the brain stayed the same, but their locations shifted slightly from day to day.

From virtual to reality

Ganguly then asked the participant to imagine himself making simple movements with his fingers, hands or thumbs over the course of two weeks, while the sensors recorded his brain activity to train the AI.

Then, the participant tried to control a robotic arm and hand. But the movements still weren’t very precise.

So, Ganguly had the participant practice on a virtual robot arm that gave him feedback on the accuracy of his visualizations. Eventually, he got the virtual arm to do what he wanted it to do.

Once the participant began practicing with the real robot arm, it only took a few practice sessions for him to transfer his skills to the real world.

He could make the robotic arm pick up blocks, turn them, and move them to new locations. He was even able to open a cabinet, take out a cu,p and hold it up to a water dispenser.

Months later, the participant was still able to control the robotic arm after a 15-minute “tune-up” to adjust for how his movement representations had drifted since he had begun using the device.

Ganguly is now refining the AI models to make the robotic arm move faster and more smoothly, and planning to test the BCI in a home environment.

For people with paralysis, the ability to feed themselves or get a drink of water would be life-changing.

Ganguly thinks this is within reach.

“I’m very confident that we’ve learned how to build the system now, and that we can make this work,” he said.