Google translates hand gestures to speech with sign language AI

An image of Google, AI, Google translates hand gestures to speech with sign language AI

Google has announced a new machine learning model for tracking hands and recognising gestures, giving once soundless sign language a voice

There are thousands of spoken languages the world over. Each has its own subtleties and nuances. Developing AI to interpret and translate those can be difficult, and even the most sophisticated translation tools struggle to do so correctly. Sentences can become jumbled, meanings are misread, and colloquialisms are mostly lost on machines. 

However, there has been a breakthrough in perceiving and translating the language of the human hand.

Google’s development of “Real-Time Hand Tracking”, which perceives hand movements and gestures, allows for direct on-device translation to speech.

The tech giant has mapped 21 3D keypoints, or coordinates, to around 30,000 real images of hands performing a variety of gestures and shapes. 

Using a complex machine learning model, they have created a mixed training schema to synchronise the data from rendered images, real-world images, and hand presence, to give a “hand present classification.”

Google hand tracking machine learning

In a blog post, Google said: “The ability to perceive the shape and motion of hands can be a vital component in improving the user experience across a variety of technological domains and platforms.”

The research and development of this machine learning algorithm could create numerous possibilities, not least for sign language understanding.

Google has not developed a stand-alone app for its algorithm but has published it open-source, allowing for other developers to integrate it into their own tech, a move which is welcomed by campaigners.

There are also potential uses for virtual reality control and digital-overlay augmented reality, or even gesture control functions in driverless vehicles and smart devices. 

Assistive tech is making strides as more and more entrepreneurs and engineers enter the market ahead of the consumer curve, with wearables and IoT devices aimed at making the lives of those with assistive needs a little easier.

An image of Google, AI, Google translates hand gestures to speech with sign language AI

Luke Conrad

Technology & Marketing Enthusiast

AI alignment: teaching tech human language

Daniel Langkilde • 05th February 2024

However, Embodied AI refers to robots, virtual assistants or other intelligent systems that can interact with and learn from a physical environment. In order to do this, they’re built with sensors that can gather data from their surroundings, with this they also have AI systems that help them analyse data they collect, and ultimately learn...

CARMA announces acquisition of mmi Analytics

Jason Weekes • 01st February 2024

CARMA announces acquisition of mmi Analytics, expanding expertise in Beauty, Fashion, and Lifestyle sectors The combined organisation is set to redefine the landscape of media intelligence, providing unparalleled expertise and comprehensive insights for PR professional and marketers in the exciting world of beauty, fashion and lifestyle.

Managing Private Content Exposure Risk in 2024

Tim Freestone • 31st January 2024

Managing the privacy and compliance of sensitive content communications is getting more and more difficult for businesses. Cybercriminals continue to evolve their approaches, making it harder than ever to identify, stop, and mitigate the damages of malicious attacks. But, what are the key issues for IT admins to look out for in 2024?

Revolutionizing Ground Warfare Environment with Software-Enabled Armored Vehicles

Wind River • 31st January 2024

Armoured vehicles which are purpose-built for mission-critical operations are reliant on control systems that provide deterministic behaviour to meet hard real-time requirements, deliver extreme reliability, and meet rigorous security requirements against evolving threats. Wind River® has the partners and the expertise, a proven real-time operating system (RTOS), software lifecycle management techniques, and an extensive track...

The need to prove environmental accountability

Matt Tormollen • 31st January 2024

We are currently in the midst of one of the most consequential energy transitions since records began. The increasing availability of clean electrons has motivated businesses in the UK and beyond to think green. And for good reason. Being environmentally conscious attracts customers, appeases regulators, retains staff, and can even gain handouts from government. The...

Fuelling Innovation in Aftermarket

Jim Monaghan • 31st January 2024

One section of the motor trade is benefitting from the cost-of-living crisis: with consumers keeping their cars for longer, independent repairers are in huge demand. But they are also under pressure. Older cars need more repairs. They require more replacement parts, tyres and fluids. With car owners looking for value and a fast turn-around, independents...

The return of the five-day office week

Virgin Media • 25th January 2024

Virgin Media O2 Business has today published its inaugural Annual Movers Index, revealing four in ten companies are back to the office full time, despite widespread travel delays and disruptions With 2023 cementing the cost-of-living crisis, second hand shopping and public transport use surged as Brits sought to save money Using aggregated and anonymised UK...