Brain computer interface turns ‘mental handwriting’ into text

0

A new brain computer interface (BCI) can convert ‘mental handwriting’ – the act of imagining you are writing on a piece of paper – into text on a screen in real time.

The groundbreaking system, developed by American researchers, uses a computer to decode handwriting movements from signals in the brain.

The signals are detected by small square electrode arrays about the size of a baby aspirin pill inserted into the brain.

Researchers say the system could allow people with paralysis who can’t use their hands or fingers to type messages quickly.

In a clinical trial, a paralyzed man used the BCI to write text on a computer screen just by thinking about the movements involved in writing.

The man was able to copy sentences and answer questions at a pace comparable to that of someone his age typing on a smartphone.

Researchers use small electrode arrays (shown) to record signals from the brain's motor cortex.  These signals can then be used to control robotic prostheses, computers or other devices.  The hope is that such a system can help restore communication and movement in people with paralysis due to injury or illness

Researchers use small electrode arrays (shown) to record signals from the brain’s motor cortex. These signals can then be used to control robotic prostheses, computers or other devices. The hope is that such a system can help restore communication and movement in people with paralysis due to injury or illness

WHAT IS BRAINGATE?

BrainGate is a brain implant system originally developed by researchers at Brown University in collaboration with biotech company Cyberkinetics.

BrainGate is based on technology to detect, transmit, analyze and apply the language of neurons.

It consists of a sensor implanted in the motor cortex of the brain and a device that analyzes brain signals.

The principle behind the BrainGate system is that brain signals are generated when brain functionality is intact, even if they are not sent to the arms, hands and legs.

With BrainGate the signals are interpreted and translated into cursor movements and activity on a screen.

This may allow someone to control a computer with thoughts, much like individuals who have the ability to move their hands use a mouse.

In particular, the system is designed to restore functionality for a limited, immobile group of persons with severe motor disabilities.

The new study is the latest phase of a clinical trial called BrainGate, led by Dr. Leigh Hochberg, a professor of neurology at Brown University in Providence, Rhode Island.

Frank Willett, a research scientist at Stanford University and the Howard Hughes Medical Institute (HHMI), led the study.

“We want to find new ways to get people to communicate faster,” said Willett.

“This new system takes advantage of both the rich neural activity registered by intracortical electrodes and the power of language models that, when applied to the neural-decoded letters, can create fast and accurate text.”

If someone is unable to move due to an injury or illness, the neural activity of the brain for walking, speaking, and grasping objects remains intact.

Researchers can therefore tap into the brain activity of people with paralysis or amputations to regain some of their lost abilities.

For their study, the researchers worked with a 65-year-old man who had a spinal cord injury in 2007 that left him paralyzed from the neck down.

Scientists implanted two of the tiny electrode arrays in the part of his brain that controls the hand and arm – the motor cortex.

Each of the electrode arrays is covered with 100 electrodes that receive signals from neurons firing into the motor cortex.

The authors instructed the man to try to write sentences as if his hand were not paralyzed, by imagining that he was holding a pen on a piece of lined paper.

Two small sets of implanted electrodes sent information from the brain region controlling the hands and arms to an algorithm, which translated it into letters that appeared on a screen

Two small sets of implanted electrodes sent information from the brain region controlling the hands and arms to an algorithm, which translated it into letters that appeared on a screen

During the exercise, a neural network (a type of machine learning) could translate attempts at handwriting movements from neural activity into text that appeared on a screen in real time.

Using this ‘brain-to-text’ system, the man answered questions by trying to write every letter of his answer.

An earlier system from the BrainGate team demonstrated in 2015, which also used sensors implanted in the motor cortex, allowed paralyzed people to select letters with a thought-driven cursor.

Pointing and clicking letters in this way allowed people to type about 40 characters per minute, the previous speed record for typing with a Brain Computer Interface (BCI).

In these new tests, the man was able to type more than twice as fast: 90 characters per minute with an accuracy of 94.1 percent, setting a new record.

This is also comparable to typical smartphone speed types in people of the same age group as the study participant (115 characters per minute).

Investigators worked with a 65-year-old man (photo) who had become paralyzed from the neck down.  The BrainGate BCI allowed him to identify letters while trying to write them in his head.  Then the system displayed the text on a screen in real time

Investigators worked with a 65-year-old man (photo) who had become paralyzed from the neck down. The BrainGate BCI allowed him to identify letters while trying to write them in his head. Then the system displayed the text on a screen in real time

This BCI is so fast because each letter evokes a very distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from the other.

In the future, researchers will Imagine trying to hand-write for text input as part of a more comprehensive system that also includes point-and-click navigation, much like that used on current smartphones, and even voice decoding.

The team wants to work with a participant with amyotrophic lateral sclerosis, a degenerative neurological condition that leads to loss of both movement and speech.

According to the BrainGate team, further demonstrations of the ‘longevity, safety and efficacy’ of the system will be needed before it can be used clinically on a large scale.

The research is published today in the journal Nature.

BCIs RESTORE SENSORY MOTOR FUNCTIONS BY TRANSLATING BRAIN SIGNALS INTO COMMANDS

Brain-computer interfaces (BCIs) are devices that provide communication paths between a user’s brain and an external device.

BCIs record brain signals, analyze them, and translate them into commands that are passed to output devices that perform the desired actions.

BCIs are not mind-reading devices, but allow their users to respond to the world by using brain signals instead of muscles.

The main purpose of BCI is to replace or restore a useful function of people with disabilities due to neuromuscular disorders, such as cerebral palsy, stroke or spinal cord injury.

BCI technology allows individuals who cannot speak and use their limbs to re-communicate or operate assistive devices for walking and manipulating objects.

.