Google translates hand gestures to speech with sign language AI

Google has announced a new machine learning model for tracking hands and recognising gestures, giving once soundless sign language a voice

There are thousands of spoken languages the world over. Each has its own subtleties and nuances. Developing AI to interpret and translate those can be difficult, and even the most sophisticated translation tools struggle to do so correctly. Sentences can become jumbled, meanings are misread, and colloquialisms are mostly lost on machines. 

However, there has been a breakthrough in perceiving and translating the language of the human hand.

Google’s development of “Real-Time Hand Tracking”, which perceives hand movements and gestures, allows for direct on-device translation to speech.

The tech giant has mapped 21 3D keypoints, or coordinates, to around 30,000 real images of hands performing a variety of gestures and shapes. 

Using a complex machine learning model, they have created a mixed training schema to synchronise the data from rendered images, real-world images, and hand presence, to give a “hand present classification.”

Google hand tracking machine learning

In a blog post, Google said: “The ability to perceive the shape and motion of hands can be a vital component in improving the user experience across a variety of technological domains and platforms.”

The research and development of this machine learning algorithm could create numerous possibilities, not least for sign language understanding.

Google has not developed a stand-alone app for its algorithm but has published it open-source, allowing for other developers to integrate it into their own tech, a move which is welcomed by campaigners.

There are also potential uses for virtual reality control and digital-overlay augmented reality, or even gesture control functions in driverless vehicles and smart devices. 

Assistive tech is making strides as more and more entrepreneurs and engineers enter the market ahead of the consumer curve, with wearables and IoT devices aimed at making the lives of those with assistive needs a little easier.

Luke Conrad

Technology & Marketing Enthusiast

eCMR: If not now, then when?

Gerry Daalhuisen • 17th July 2024

There have been several unexpected pit stops on the road to eliminating paper-based processes in logistics. But, is paper finally set to be a thing of the past?

Tackling Tech Debt

Wes van den Berg • 16th July 2024

5 years ago if you were a CIO without a cloud strategy you’d likely be out of a job. But making decisions in haste might mean businesses ended up with technology they regret, that doesn’t deliver on the promised value.

Laying the foundations for global connectivity

Waldemar Sterz • 26th June 2024

With the globalisation of trade, the axis is shifting. The world has witnessed an unprecedented rise in new digital trade routes that are connecting continents and increasing trade volumes between nations. Waldemar Sterz, CEO of Telegraph42 explains the complexities involved in establishing a Global Internet and provides insight into some of the key initiatives Telegraph42...

IoT Security: Protecting Your Connected Devices from Cyber Attacks

Miro Khach • 19th June 2024

Did you know we’re heading towards having more than 25 billion IoT devices by 2030? This jump means we have to really focus on keeping our smart devices safe. We’re looking at everything from threats to our connected home gadgets to needing strong encryption methods. Ensuring we have secure ways to talk to these devices...