Jawaid Shabir1, Sobia Shabir2*, Ruhan Uddin Shaikh3, Rida Fatima4, Syeda Tehreem Qamar5, Maria Khan6, Zoha Tariq7
1Assistant Professor(Computer Engineering)Sir Syed University of Engineering and Technology
2M.S. Scholar(Biomedical Engineering) Sir Syed University of Engineering and Technology
3B.S. Scholar(Biomedical Engineering) Sir Syed University of Engineering and Technology
4M.S. Scholar(Biomedical Engineering) Sir Syed University of Engineering and Technology
5BS biomedical engineering(Sir Syed University of Engineering & Technology)
6MS Computer Science, Information Technology (NED University)
7MS in information security( NED University)
*Corresponding Author: Sobia Shabir, M.S. Scholar(Biomedical Engineering) Sir Syed University of Engineering and Technology
Abstract
Numerous individuals in our nation are hard of hearing and dumb. How frequently do we run over these individuals speaking with the ordinary world? The correspondence amongst hard of hard-of-hearing individuals and dump with the ordinary individual is a major issue compared with the correspondence between the visually impaired and the typical visual individual. Gesture-based communication is the nonverbal frame. Our essential thought is to change over voice to content and content to voice in our local dialect so individuals can comprehend them effectively. This can be fulfilled by methods for instruments like Java.
Keywords: Android Studio, Google API, MySQL, firebase.
Introduction
Hard of hearing individuals ordinarily utilize gesture-based communications keeping in mind the end goal to speak with each other. In this correspondence framework, hard of hearing individuals are not ready to speak to their thoughts or messages which they need to state to other individuals. In this day and age, innovation has been created so quickly and introduces each activity in a computerized format then into pictures, recordings or sound arrangement. To make their lives further developed, an application should be produced which gives a chance to learn new things and an opportunity to get acquainted with new innovations. Hard of hearing individuals are generally denied of typical correspondence with other individuals in the general public. It has been watched that they discover it extremely troublesome now and again to communicate with ordinary individuals with their signals, as only a not very many of those are perceived by the vast majority. Gesture based communication is the essential methods for correspondence in the not too sharp group. For the Deaf people group, we attempt to enhance their personal satisfaction by creating frameworks that can enable them to discuss better with whatever remains of the world and among themselves. Our System gives a learning background to find out about gesture-based communication. It likewise gives correspondence in which input is given as discourse or content. The Speech Recognition innovation changes over discernible language into ordinary messages and voice in advanced mobile phones.
Through this work, we can without much of a stretch tackle the majority of their concern in one application. In this manner, the present works intend to:
1. Help the tragically challenged to associate more with ordinary individuals.
2. It can likewise help them in imparting in their local dialect.
3. Group correspondence is additionally present in our application.
Literature Review
G.V.S. Subhaashini, S.Divya, S. DivyaSuganya, Tharani Vimal proposed a Speech-to-Sign and Sign to-Speech technology are used. When the hearing party inputs voice or text, it will be converted into sign language video on the deaf end. This speech-to-sign conversion is achieved by Mimix technology, which translates spoken and written words into sign language with a 3D character. By using this application deaf person can easily interact with normal person anywhere, and he can also use this application for mobile sign translation using Video Relay Service. Also using UTF-7 he can communicate in daily activates without dialing a number. [1]
Patil Diksha, Jadhav Ashwini, Chavan Amruta and Nikam Shrutika proposed a system which is being developed mainly in two parts which consists of speech recognition and conversion of speech into image form. The first part receives the input in the form of a speech. The speech is captured through a speech recognition system as an analogue signal.
Then it is digitized and translated into sign language. The second part of the system converts the sign language to avatars which are displayed on the computer screen for the user to see. The system will refer to two sets of databases. The first database, which we call the Sign and Speech database will contain all the images of the alphabets and words. While the second database called the Avatar database will contain the equivalent avatar for each alphabet and words.[2]
Gayathri1, Dr.A. Sasi Kumar2 states that using this application we paved a way for the deaf person who can easily interact with normal person anywhere. This project also supports Automatic translation, automotive speech recognition, and Speech-to-sign transmission. Our proposed system includes a variety of technologies. It consists of two main parts hardware and software. In hardware parts we required phone speaker. In software, we mainly consider Outfit-7 (which is used in tomcat application) and Video Relay Service (VRS). All these parts can be brought together in an integrated system. In this system, we implement outfit-7 in the VSR application. Outfit-7 is an application for the mobile phone, with the software which will convert everything we say into a high-pitched voice. Without dialing a number, we can use this application. [3]
Ujwala D Rode Stela R. Wavika states that this system improves communication with the deaf and dumb using flex sensor technology. Flex sensors are placed on hand gloves for the use of deaf and dumb people. Flex sensor ‘s resistance changes according to the flexion experienced. Sensors in the glove pick up gestures and transmit that arduino (AT MEGA328). This Gestures which come from arduino board the Bluetooth module send that sign to Android phone only when android phones Bluetooth is enable. By using the mobile App incoming message will convert to voice. Here device recognizes alphabets only. When the normal people want to communicate with deaf people, there is also an App that converts speech to text data. [4]
Dr. D. Y. Patil, Savitribai Phule University of Pune, India Loss of hearing and speech can cause people to become isolated and lonely, having worse effect on both their social and working life. With advancement of science and technology many techniques have been developed not only to minimize the problem of deaf and dumb people but also to implement it in different fields. Alexander Dreyer Johnson et al. Proposed a Touch-based mobile system for the visually impaired. They introduced the idea of using finger touch mobile devices to help the visually impaired. [5]
1V.Purushotham Vijay Naidu 2M.R.Sai Hitesh, 3T.Dhikhi states that there are many applications available in the market to help the Deaf and Dumb people to interact with the world. Voice-based email and chatting systems are available to communicate with each other by the Deaf and Dumb. This helps to interact with persons by Deaf and Dumb people.
Android application has shown a dramatic improvement in their practicality to a degree where it’s currently attainable to possess a mobile phone executes Java programs. As a result, cellular users throughout the globe square measure currently ready to scan their system is a combination of sound and haptic. It depends upon haptic and voice feedback; this application includes to conversion voice to text based and text to voice-based interaction approach. Our work helps in improving the communication with the deaf and dumb. Text- to-Voice and Voice-To-Text technology convert translation on smart phones. Interaction between normal people with Deaf and Dumb person is very difficult because of communication problems. This application has a lot of benefits for all Deaf and dumb peoples. This people easily communicate with other people without any hesitation. This application gives your chances to show your abilities to anyone. Every voice and text record which is stored in the database. A database has been created from the various domain words and syllables. [6]
Anbarasi Rajamohan, Hemavathy R., Dhanalakshmi M. stats that Conversation between the deaf and dumb and an ordinary individual have dependably been a testing assignment. The project expects to encourage individuals by methods for a glove based deaf and dumb conversation interpreter framework. The glove is inside outfitted with five flex sensors, material sensors and accelerometer. For every particular signal, the flex sensor creates a relative change in opposition furthermore, accelerometer measures the introduction of hand. The preparation of these hand motions is in Arduino. The glove incorporates two methods of task preparation mode to advantage each client and an operational mode. The link of letters to frame words is additionally done in Arduino. Moreover, the framework additionally incorporates a content to discourse transformation (TTS) square which deciphers the coordinated signals i.e. content to voice yield. [7]
Methodology
The methodology we will follow consists of exceptional sections and the most tough factor in this project would be the voice synchronization to textual content and text to voice synchronization by using end to end communication method (TCP) feature when the users are distant from each other and head-to-head communication at the same place by using this mobile application. Specific stages are given underneath.
A. Design Phase
It's far an end-to-end mobile application to be a direction between two users after registering an account with critical biological records. Communication can have, between users/multi-user by way of a runtime text to audio converter and audio to audio (subtitles i.e text).
B. Implementation Phase
Mobile application that might get those voices eventually work on receiver mobile devices for a spare arrangement for sound What’s more video (audio with subtitles). It’s much an end-to-end communication by mobile application requisition as an approach on heading the middle of two clients after registering a record for discriminating living records. Correspondence can be written, between users/multi-user Toward method for runtime quick with sound converter Also sound should audio (subtitles i.e text).
Figure 1: (System Diagram)
Flow Of The Proposed System
The following diagram represent the overall flow of the system.
Figure 2: (Deaf Can Listen And Dumb Can Speak) An overview diagram of a connection between Deaf And Dumb Chatter box
Features Of DND Chatterbox
The DND chatterbox contains following features
A. Dumb Module
In this module, the dumb execute type the textual content in the textual content container so much is his/her thinking as he/she needs to speak, or alter such of an audio version, so each person may hear without difficulty what the dumb desires in imitation of say.
B. Deaf Module
In this module deaf may hear by surely ON the microphone to record/scan the audio/voice about the manager then put such among a video format audio with subtitles.
C. TCP Conversation Module
1. Dumb to Deaf: Dumb first type the text in the text box and convert it format of video (audio with subtitles) and save it in their mobile devices which will be send to the deaf by internet connection through mobile application (DnD chatter- box) where on the other hand deaf receive the video (audio with subtitles).
2. Deaf to Dumb: Deaf record his/her audio and convert it into video (audio with subtitles) and send it to dumb.
Results And Discussion
The proposed technology was tested using three different ways. First test was performed inside the SDK to check the functionality. The second test was achieved by launching the application on different phone devices. The third test was executed by asking 10 deaf persons and 15 normal volunteers to use the application. In the first verification test, the application was simulated to test its functionality of the application. All tests were performed for code verification and library checking on the application without any errors or bugs. No false positives or negatives were recorded. The second test was performed to ensure that the application is working correctly on different screen sizes and resolutions. It was tested on various devices from different manufacturers, such as: Sony Xperia Z2, Samsung Galaxy S4, Samsung Galaxy S6, and LG G2. In this test, the application worked properly on the different devices. For the third verification test, we used usability testing that gives direct input on how real users use the system. We evaluated our application by testing it on real deaf users. 10 deaf or dumb people and 15 normal volunteers use the application for more than a month. We carefully created scenarios, or realistic situations, for testing purposes. The subjects were then tested to see if their signs are understood effectively and efficiently. The questionnaire contains five simple questions with a scale of five answers (1 = Poor, 2 = Fair, 3 = Good, 4 = Very Good, 5 = Excellent). The questions are designed to investigate if the application is easy to use, has clear instructions, is helpful, and has a short response time. The last question was intended to find if the subjects are willing to use it daily.
Conclusion
In this paper, we presented a mobile application to help deaf and dumb people in their daily activities. The key feature of this application is engaging the Urdu language as a medium of communication to learn all the sign language terms. The proposed assistive applications allow normal people to communicate with the targeted people without having any previous knowledge of sign language. The new application was tested on real deaf, dumb, and normal people. All the deaf and dumb found the application valuable, and most of them wanted to use it on a regular basis.
Acknowledgement
We want to thank all those people who helped us during this journey. Without everyone's support, we could not have accomplished our work. We express our sincere gratitude to Mr. jawaid Shabbir (Department of Computer Engineering) for his amazing support and guidance, throughout the course of the present work. And last but not the least we are extremely grateful to the whole faculty and administration of Sir Syed University for providing us all the facilities that were mandatory, without which this project could not have been a success.
References
- Subhaashini GVS, Divya S, Divyasuganya S, Vimal T (2015) Ear Hear Android Application For Specially Abled Deaf People. UG Scholar, Assistant Professor Department Of Information Technology St.Peters College Of Engineering And Technology, Chennai, India. Inter J Of Com Scie And Engine Commun. 3(3).
- Diksha P, Ashwini J, Amruta C, Shrutika N (2016) Android Application for Deaf People Department Of Computer Engi- neering, Matoshri College of Engineering and Research Centre. Imperial J of Interdisciplinary Res (Ijir). 2(3).
- Gayathri A, Kumar AS (2017) Sign Language Recognition for Deaf and Dumb People Using Android Environment 1pg Scholar, Dept. Of It, School Of Computing Sciences 2associate Professor, Dept. Of It, School Of Computing Sciences Vels University, Chennai, India. Int J Of Current Engin And Scie Res. (Ijcesr).
- Ujwala D, Stela R, Wavika R (2014) Sign Recognition For Dump And Deaf People Using Android App Computer Department Shatabdi Institute of Engineering and Research. Agaskhind Nasik. Com Dep Shatabdi Insti of Engin and Res.
- Dr. D. Y. Patil School of Engineering and Technology. Savitribai Phule University of Pune, India.
- V.Purushotham Vijay Naidu, M.R.Sai Hitesh, T.Dhikhi.
- Rajamohan A, Hemavathy R, Dhanalakshmi M (2013) Deaf- Mute Communication Interpreter Department of B.M.E., Sri Sivasubramania Nadar College of Engineering, Chennai, Tamil Nadu, India.