A Support Vector Machine Implementation for Sign Language Recognition

Quick Update:

I have not had enough time to do a comprehensive update on all the recent developments around this project, here is a quick summary before delving into the technical implementation:

  • The project recently emerged the grand winner of the hardware trailblazer award at the American Society of Mechanical Engineers(ASME) global finals in New York. You can read more about this here. It competed against other impressive innovations from America, India and the rest of the world.
  • Sign-IO also placed 2nd runners-up at the Royal Academy of Engineering Leaders in Innovation Fellowship in London.
  • Below are a few images of the prototyping efforts so far(top being most recent version)

The current iteration is refined for portability and appears as shown below:

Pretty neat, right?
Pretty neat, right?:-)

IMG_20171008_200341_2

blog1
The previous version

Currently, more than 30 million people in the world have speech impairments and thus to communicate have to use sign language resulting in a language barrier between sign language and non-sign language users. This project explores the development of a sign language to speech translation glove by implementing a Support Vector Machine(SVM) on the Intel Edison to recognize various letters signed by sign language users. The data for the predicted signed gesture is then transmitted to an Android application where it is vocalized.

The Intel Edison is the preferred board of choice for this project primarily because:

  1. The huge and immediate processing needs of the project as Support Vector Machines are a machine learning algorithms that require a lot of processing power and memory. In addition to this we need our output in real-time.
  2.  The inbuilt Bluetooth module on the Edison is used to transmit the predicted gesture to the companion Android application for vocalization.

I have also published this project on the Intel Developer Zone site.

The project code can be downloaded from here

Below is the short project video demo from a while back:

1. The Hardware Design

The sign language glove has five flex sensors mounted on each finger to quantify how much a finger is bent.

Continue reading

Predicting user activity using an accelerometer on Intel based wearables

This project describes how to recognize certain types of human physical activities using acceleration data generated from the ADXL345 accelerometer connected to the Intel Edison board.

I have also published this project on the Intel Developer Zone site.

Human activity recognition has a wide range of applications especially in wearables. The data collected can be used monitoring patient activity, exercise patterns in athletes, fitness tracking and so on.

We will be using support vector machine learning algorithm to achieve this. The project is implemented using the LIBSVM library and done separately in both Python and node.js.

The set of activities to be recognized are running, walking, going up/down a flight of stairs and resting. We collect the accelerometer data over a set of intervals, extract the features which in this case are the acceleration values along the x,y and z axes. We then use this data for building a training model that will do the activity classification.

The project code can be downloaded from here

Hardware Implementation.

The diagram below shows the pin connection for the ADXL345 to the Intel® Edison board.

adxl345pinconnection

Part 1. Python Implementation.

Setting up LIBSVM

Download the LIBSVM library and transfer the  LibSVM  zipped folder to the Intel® Edison board root directory using WINSCP. Then extract it by running:

tar –xzf libsvm-3.21.tar.gz

Run  make in libsvm-3.21 directory

Continue reading