Digital Assisted Communication

The communication with the deaf community can prove to be very challenging without the use of sign language. There is a considerable difference between sign and written language as they differ in both syntax and semantics. The work described in this paper addresses the development of a bidirectional translator between several sign languages and their respective text, as well as the evaluation methods and results of those tools. A multiplayer game is using the translator is also described on this paper. The translator from sign language to text employs two devices, namely the Microsoft Kinect and 5DT Sensor Gloves in order to gather data about the motion and shape of the hands. This translator is being adapted to allow the communication with the blind as well. The Quantitative Evaluation Framework (QEF) and the ten-fold cross-validation methods were used to evaluate the project and show promising results. Also, the product goes through a validation process by sign language experts and deaf users who provide their feedback answering a questionnaire. The translator exhibits a precision higher than 90% and the projects overall quality rates are close to 90% based on the QEF.