A Support Vector Machine Implementation for Sign Language Recognition

Quick Update:

I have not had enough time to do a comprehensive update on all the recent developments around this project, here is a quick summary before delving into the technical implementation(also the LSTM implementation is in the works).

  • The project recently emerged the grand winner of the hardware trailblazer award at the American Society of Mechanical Engineers(ASME) global finals in New York. You can read more about this here. It competed against other impressive innovations from America, India and the rest of the world.
  • Sign-IO also placed 2nd runners-up at the Royal Academy of Engineering Leaders in Innovation Fellowship in London.
  • Below are a few images of the prototyping efforts so far(top being most recent version)

The current iteration is refined for portability and appears as shown below:

Pretty neat, right?
Pretty neat, right?:-)


The previous version

Currently, more than 30 million people in the world have speech impairments and thus to communicate have to use sign language resulting in a language barrier between sign language and non-sign language users. This project explores the development of a sign language to speech translation glove by implementing a Support Vector Machine(SVM) on the Intel Edison to recognize various letters signed by sign language users. The data for the predicted signed gesture is then transmitted to an Android application where it is vocalized.

The Intel Edison is the preferred board of choice for this project primarily because:

  1. The huge and immediate processing needs of the project as Support Vector Machines are a machine learning algorithms that require a lot of processing power and memory. In addition to this we need our output in real-time.
  2.  The inbuilt Bluetooth module on the Edison is used to transmit the predicted gesture to the companion Android application for vocalization.

I have also published this project on the Intel Developer Zone site.

The project code can be downloaded from here

Below is the short project video demo from a while back:

1. The Hardware Design

The sign language glove has five flex sensors mounted on each finger to quantify how much a finger is bent.

Continue reading