Inspiration
There are 466 million persons in the world with disabling hearing loss (6.1% of the world's population) , I have created this app for them and their family so that they can communicate easily. This will allow a person with hearing loss and person who doesn't knows Sign language to communicate to each other easily. We were also inspired from Hana's ASL workshop.

What it does
It can do two things:

Take Voice from User and Convert it to Sign Language in form of a GIF
It will Take Video from User and convert the Sign Language in Video to Voice using Machine Learning
How we built it
We divided the project into two parts

VOICE/TEXT TO SIGN LANGUAGE CONVERSION:
Scraping Data from Giphy using Chrome Extension
Then filtered the gif files and added names to it
Also added gif files of single alphabets
Took Voice/Text input from user and split into words and checked if it is present in the GIF filenames. If it is not present then use the Alphabet GIFs for making up words
Finally Displayed it onto Tkinter App
asl1

SIGN LANGUAGE TO VOICE/TEXT CONVERSION:
Used the ASL Dataset on Kaggle of Alphabets
Created a CNN algorithm in Tensorflow and trained the model for small data
Used Live Webcam feed of user hand and predicted the Alphabet from a Region of Interest
Finally Displayed it onto Tkinter App
asl2

Challenges we ran into
Tried using Scrapy and Selenium to scrape the Gif files but were unsuccessful
Creating GUI on Tkinter was hard as we were using it for first time
Accomplishments that we're proud of
Successfully created a Multifeatured Desktop App
Got Accuracy of 95% in predicting Sign Language
What we learned
Using Tkinter
Team Work and Time Management
Scraping Data from websites
What's next for Two Way Sign Language Translate
Adding Threading to Tkinter App
Creating and Deploying a Web Based Version using Flask and GoDaddy Domain
Adding support to Words in ASL Detection
Using Google's Will SDK to improve Accuracy

свернуть