Sign Language Recognition on Google Glass
School: Los Alamos High
Area of Science: Computer Vision
Interim: Problem Definition:
Modern advancements in image processing and analysis offer new avenues for expression and understanding of the world in software. The communication barrier that exists between deaf Americans who use sign language, for example, can be overcome by applications that are able to identify and translate sign language to spoken or written English. The primary issue with these programs is that they difficult to apply when they run exclusively on immobile hardware.
In recent years, new head mounted hardware (especially Google Glass) offer innovative and interesting applications for software that were not previously available. We've been looking into OpenCV, and we want to use their machine learning and signal processing libraries. Specifically, we’ll be using Hidden Markov Models and Histogram of Oriented Gradient methods to achieve our goals. We intend to start by isolating people, and specifically their hands, from crowded scenes. We will almost certainly be using Java to interface with the glass, but will most likely be using c for the computations. There has been some discussion of running calculations on a substantially more powerful handheld device (such as a smartphone) and using bluetooth to communicate with the glass.
Progress to Date:
At the start of this project, we were already somewhat familiar with Hidden Markov Models and Histograms of Oriented Gradients, as well as the basic capabilities of OpenCV, but we chose to start by further exploring these methods as well as researching whether the methodologies we chose were the most optimal with which to solve our problem.
We have also built and tested some basic programs on the Google Glass in order to fully test its capabilities and set up for more advanced programs.
We hope to be able to reliably identify hands in crowded scenes and identify the gestures on their hands. Once the gestures are identified their meaning will be translated from ASL to English on the glass screen.
Sponsoring Teacher: Adam Drew
Mail the entire Team