Google Pixel 11 Translates American Sign Language into Text with AI

Google has unveiled a new feature for its Pixel 11 smartphone that allows users to translate American Sign Language (ASL) into text. The technology, called SL2T, uses AI to track the signer's entire upper body and translates their movements into written English.

The model was trained on over 100,000 hours of data from more than 50 sign languages, including ASL. It can also understand one-handed signing and signing by left-handed users. However, Google warns that SL2T is not a replacement for human interpreters and should only be used for low-stakes communication.

SL2T uses an on-device computer vision system to track 130 points across the signer's face, body, and hands. The raw footage is discarded, while the coordinates are sent to Google's servers for translation. The company does not retain logs of what users sign or the text produced unless they agree to participate in a model evaluation study.

Google plans to continue working with its AI Sign Language Advisory Committee as it expands the technology to additional sign languages and eventually generates sign language as well.