Activity detection in conversational sign language video for mobile telecommunication

Neva Cherniavsky, Richard E. Ladner, Eve A. Riskin

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

The goal of the MobileASL project is to increase accessibility by making the mobile telecommunications network available to the signing Deaf community. Video cell phones enable Deaf users to communicate in their native language, American Sign Language (ASL). However, encoding and transmission of real-time video over cell phones is a powerintensive task that can quickly drain the battery. By recognizing activity in the conversational video, we can drop the frame rate during less important segments without significantly harming intelligibility, thus reducing the computational burden. This recognition must take place from video in real-time on a cell phone processor, on users that wear no special clothing. In this work, we quantify the power savings from dropping the frame rate during less important segments of the conversation. We then describe our technique for recognition, which uses simple features we obtain "for free" from the encoder. We take advantage of the conversational aspect of the video by using features from both sides of the conversation. We show that our technique results in high levels of recognition compared to a baseline method.

Original languageEnglish
Title of host publication2008 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008
DOIs
StatePublished - 2008
Event2008 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008 - Amsterdam, Netherlands
Duration: 17 Sep 200819 Sep 2008

Publication series

Name2008 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008

Conference

Conference2008 8th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2008
Country/TerritoryNetherlands
CityAmsterdam
Period17/09/0819/09/08

Fingerprint

Dive into the research topics of 'Activity detection in conversational sign language video for mobile telecommunication'. Together they form a unique fingerprint.

Cite this