Record Details

A Novel Method for Lip Movement Detection using Deep Neural Network

NOPR - NISCAIR Online Periodicals Repository

View Archive Info
 
 
Field Value
 
Title A Novel Method for Lip Movement Detection using Deep Neural Network
 
Creator Srilakshmi, Kanagala
R, Karthik
 
Subject Attention
CNN
EfficientNet
Lip movement
LSTM
MIRACL-VC1
 
Description 643-650
Recognition of Lip movements has become one of the most challenging tasks and has crucial applications in the contemporary scenario. It is the recognition of the speech uttered by individual using visual cues. Visual interpretation of lip movement is especially useful in scenarios like video surveillance, where auditory signals are either not available or too noisy for interpretation. It is also useful for hearing-impaired individuals where audio signal is of no use. Many developments have taken place in this nascent field using various deep learning-based techniques. This research does analysis over various state-of-the-art deep-learning models on MIRACL-VC1 dataset. This study also aims to find out the optimal baseline architecture suitable for building a new model with high accuracy for lip movement detection. The models are trained from scratch over the pre-processed MIRACL-VC1 dataset consisting of small-size images. Experimental observations with state-of-the-art deep learning models indicate that EfficientNet B0 architecture yielded an accuracy of 80.13%. Thus, EfficientNet B0 is further utilized as baseline deep architecture to design a customized model for effective detection. This research proposes an attention based deep learning model combined with Long Short-Term Memory (LSTM) layer having EfficientNet B0 as the backbone architecture. The proposed model yielded an accuracy of 91.13%.
 
Date 2022-06-06T06:10:24Z
2022-06-06T06:10:24Z
2022-06
 
Type Article
 
Identifier 0975-1084 (Online); 0022-4456 (Print)
http://nopr.niscpr.res.in/handle/123456789/59854
 
Language en
 
Publisher NIScPR-CSIR, India
 
Source JSIR Vol.81(06) [June 2022]