Meta envisions replacing keyboard and mouse with smart wristband

Technology company Meta has said "the wrist is the key" to the future of human-computer interaction after its engineers created a gesture-sensing wristband prototype.

Detailed in the scientific journal Nature, the wristband picks up on the tiny, barely perceptible movements of muscles at the wrist to read a variety of hand gestures, ranging from tapping the thumb against different fingertips to writing out letters.

This muscle-reading technology is called surface electromyography (sEMG), and Meta believes a wrist-worn device harnessing it has the potential to overtake the keyboard, mouse and touchscreen as our primary way of interacting with computers.

As a control scheme it is less robotic, more intuitive and "inherently more human" than current offerings, the company has written in a blog post.

It is also more inclusive, Meta says, as it works by reading signals that are still present after spinal cord injuries and can also be used by people with tremors who struggle with handheld devices.

Meta developed the wristband prototype in its Reality Labs unit, which focuses on virtual and augmented reality and has previously developed the Orion and Ray-Ban Meta glasses.

After previewing the wristband at the same time as the Orion project last year, the team has gone on to hone the prototype, getting it to the point where it can be used "out of the box" for simple gestures, despite differences in human physiology and behaviour.

The team has now demonstrated the technology in an article in the peer-reviewed journal Nature and in a video (shown top) that shows the sEMG interface recognising gestures including tapping, swiping and pinching.

"We believe this is the first high-bandwidth neuro-motor interface that generalises across people," narration on the video claims.

The wristband also recognises letters as the test subject handwrites them. For greater accuracy, this feature is calibrated to the individual user.

Meta imagines the wristband would work in conjunction with a headset or glasses like Orion, allowing users to carry out complex actions within an immersive digital environment.

"It's a future we've been exploring at Reality Labs for years," Meta said in its blog post. "Based on our findings, we believe that surface electromyography at the wrist is the key to unlocking the next paradigm shift in human-computer interaction."

The wristband works through metal contacts on the skin that detect muscle activity and deep learning models that decode the signals and transform them into computer inputs.

The gesture-based system is similar to what can be achieved using a camera or controllers with acceleration sensors, but Meta points out that these stop working if the line of sight is obscured.

There is also an obvious advantage over a neural implant, in that its not invasive, and over voice control in that it is not disruptive to others.

"It's convenient, simple and natural to use — and it works in situations where alternatives like voice interactions may be impractical or undesirable, like sending a private message out in public," said Meta.

Meta has said it will release over 100 hours of its sEMG recordings to the scientific community in hopes of accelerating research in the field.

In the Nature article, the authors also set out design rules and best practices that they recommend for hardware, data requirements and modelling.

Meta rebranded from Facebook in 2021, when it was focused on developing the metaverse. It has since pivoted to emphasise augmented reality, virtual reality and artificial intelligence (AI).

Orion and the sEMG wristband are Meta's answer to a question that has gripped Silicon Valley – whether the rise of AI is the death knell for the smartphone.

Several companies are racing to come up with a device that will deliver the advanced functionality of AI but without the distraction of screens.

The post Meta envisions replacing keyboard and mouse with smart wristband appeared first on Dezeen.