Google's AI Breakthrough Translates ASL into Written Text for Deaf Users

Google has released a new sign-to-text feature in Gboard and Live Transcribe across the Pixel 11 lineup, allowing users to translate American Sign Language (ASL) directly into written text. This technology was built by Deaf people for Deaf people, with input from the AI Sign Language Advisory Committee of Deaf organizations and subject-matter experts.

The model was trained on over 100,000 hours of data spanning more than 50 sign languages, including ASL. It can handle one-handed signing, which is essential when holding a phone with one hand, and left-handed signers. The technology tracks 130 points across the signer's face, body, and hands to convert video into coordinates.

This innovation comes after a long history of well-meaning but misguided attempts to address ASL. For instance, the sign language glove idea has resurfaced repeatedly, despite Deaf linguists explaining that it only captures hand movements and ignores facial expressions and other crucial aspects of the language.

The recognition of ASL as a complex language with its own grammar is groundbreaking. Chrissy Marshall, a Deaf writer and director, notes that this technology 'recognizes computationally as the complex language it is: from handshapes and classifiers to spatial grammar, facial expression, and non-manual markers.'

Google emphasizes that SL2T is an assistive input tool for low-stakes communication only. It does not fulfill legal requirements for reasonable accommodations and should not be used to replace qualified human interpreters or sidestep accessibility obligations.