Record Details

Sign Language Recognition using Deep CNN with Normalised Keyframe Extraction and Prediction using LSTM

NOPR - NISCAIR Online Periodicals Repository

View Archive Info
 
 
Field Value
 
Title Sign Language Recognition using Deep CNN with Normalised Keyframe Extraction and Prediction using LSTM
 
Creator P, Jayanthi
Sathia Bhama, Ponsy R K
Madhubalasri, B
 
Subject Deaf-mute people
Gesture recognition
Indian sign language
Relationship signs
Signer
 
Description 745-755
Sign Language Recognition (SLR) targets interpreting the signs so as to facilitate communication between hearing or
speaking disabled people and normal people. This makes communication between normal people and signers effective and
seamless. The scarcely available key information regarding the gestures is the key to recognise the signs. To implement
continuous sign language gesture recognition, gestures are identified from the video using Deep Convolutional Neural Network.
Recurrent Neural Network- Long Short-Term Memory verifies the semantics of the gesture sequence, which eventually will be
converted into speech. The problem of constructing meaningful sentences from continuous gestures inspired the proposed
system to develop a model based on it. The model is designed to increase the effectiveness of the classification by processing
only the principal elements. The keyframes are identified and processed for classification. Validation of sentences can be done
O(N). The sentences are converted into voiceover to have elegant communication between impaired and normal people. The
model obtained an accuracy of 89.24% while training over Convolutional Neural Network to detect gestures and performed
better than other pre-trained models and an accuracy of 89.99% while training over Recurrent Neural Network- Long Short-
Term Memory to predict the next word using grammar phrases. This keyframe-to-voice conversion, forming proper sentences,
enthrals people to have harmonious communication.
 
Date 2023-07-06T04:51:20Z
2023-07-06T04:51:20Z
2023-07
 
Type Article
 
Identifier 0022-4456 (Print); 0975-1084 (Online)
http://nopr.niscpr.res.in/handle/123456789/62270
https://doi.org/10.56042/jsir.v82i07.2375
 
Language en
 
Publisher NIScPR-CSIR,India
 
Source JSIR Vol.82(07) [July 2023]