American Sign Language (ASL) has long enabled real-time conversations for English-speaking people who are deaf and hard-of-hearing. But discussions often face significant lags when one or more conversants aren’t fluent in the language system. But by combining deep learning artificial intelligence and micro-sonar technologies, researchers at Cornell University are developing a new wearable to help overcome the communication barriers. With further refinement, SpellRing may one day facilitate entire conversations regardless of your ASL comprehension skills.
ASL’s earliest iterations developed in the early 18th century at the American School for the Deaf in Hartford, Connecticut. Today, around 400,000 people in the US utilize modern ASL, including a large number of children of deaf adults (CODA). Like any language, ASL often takes years of education and practice to reach fluency. Given that the majority of Americans don’t regularly occupy spaces requiring it, however, the language still remains mostly relegated to populations that are deaf and hard-of-hearing. In the meantime, technological innovations haven’t caught up with them.
“Many other technologies that recognize fingerspelling in ASL have not been adopted by the deaf and hard-of-hearing community because the hardware is bulky and impractical,” Hyunchul Lim, a Cornell information science doctoral student, said in a university profile on March 17. “We sought to develop a single ring to capture all of the subtle and complex finger movement in ASL.”
Lim and his colleagues previously worked on similar inventions through Cornell’s Smart Computer Interfaces for Future Interactions (SciFi) Lab, including interpretational tools for facial expressions, virtual reality hand poses, and silent speech recognition.
SpellRing builds off a previous iteration called Ring-a-Pose and relies on multiple inputs to analyze, interpret, and translate ASL fingerspelling gestures. The principle component is a quarter-sized 3D-printed ring casing that contains a small microphone and speaker, and is worn around the thumb. When the user begins fingerspelling, the microphone emits inaudible soundwaves that are subsequently detected by the microphone as a miniature gyroscope measures the hand motions. Meanwhile, a computer featuring a deep-learning algorithmic program analyzes and translates the resultant sonar images into individual letters in real-time on a computer screen.
Researchers trained SpellRing with the help of 20 experienced and novice ASL signers as they spelled out over 20,000 words. Depending on length and difficulty, SpellRing’s accuracy eventually ranged from 82–92 percent.
“There’s always a gap between the technical community who develop tools and the target community who use them. We’ve bridged some of that gap,” said Cheng Zhang, an assistant professor of information science paper co-author.
Despite the advances, SpellRing’s designers know it is only an early phase. The wearable is currently limited to fingerspelling. ASL relies on a wider set of upper body movements, facial expressions, and other physicalities, and has more than 4,000 word signs.
“Fingerspelling, while nuanced and challenging to track from a technical perspective, comprises but a fraction of ASL and is not representative of ASL as a language,” said Jane Lu, study co-author and a linguistics doctoral student. “We still have a long way to go in developing comparable devices for full ASL recognition, but it’s an exciting step in the right direction.”
Moving forward, the team plans to expand on SpellRing’s capabilities to adapt the micro-sonar system for eyeglasses that assess a user’s face and upper body.
More deals, reviews, and buying guides
