Sign language recognition by image analysis /

Download
2015
Büyüksaraç, Buket
The Sign Language Recognition (SLR) Problem is a highly important research topic, because of its ability to increase the interaction between the people who are hearing-impaired or impediment in speech. However there are several limitations of the existing methods. Most applications need different necessities like making the user wear multi-colored or sensor based gloves or usage of a specific camera. We propose a simple but robust system that can be used without the need of any specific accessories. The proposed system consists of three main steps. First we apply segmentation to the face and hand region by using Fuzzy C-Means Clustering (FCM) and Thresholding. FCM is a clustering technique which employs fuzzy partitioning, in an iterative algorithm. After the face and hands are segmented, the feature vectors are extracted. The feature vectors are chosen among the low level features such as the bounding ellipse, bounding box, and center of mass coordinates, since they are known to be more robust to segmentation errors due to low resolution images. In total there are 23 features for each hand. After the feature vectors are extracted, they are used for recognition with discrete Hidden Markov Model (HMM). Recognition stage is composed of two stages, namely training and classification. The Baum Welch algorithm is used for HMM training. In classification part the likelihood of each HMM is calculated and the HMM with the highest likelihood is chosen. In order to measure the success rate of the system, user-dependent and independent tests were conducted for 10 Turkish Sign Language gesture and the system is shown to be working with 85.8% accuracy in the user independent case and 100% in user dependent case.

Suggestions

Sign Language Recognition By Image Analysis
Buyuksarac, Buket; Bulut, Mehmet Mete; Akar, Gözde (2016-05-19)
The Sign Language Recognition (SLR) Problem is a highly important research topic, because of its ability to increase the interaction between the people who are hearing-impaired or impediment in speech. We propose a simple but robust system. The proposed system consists of three main steps. First we apply segmentation to the face and hand region by using Fuzzy C-Means Clustering (FCM) and Thresholding. FCM is a clustering technique which employs fuzzy partitioning, in an iterative algorithm. After the face a...
SPEECH DETECTION ON BROADCAST AUDIO
Zubari, Unal; Ozan, Ezgi Can; Acar, Banu Oskay; Çiloğlu, Tolga; Esen, Ersin; Ates, Tugrul K.; Onur, Duygu Oskay (2010-08-27)
Speech boundary detection contributes to performance of speech based applications such as speech recognition and speaker recognition. Speech boundary detector implemented in this study works on broadcast audio as a pre-processor module of a keyword spotter. Speech boundary detection is handled in 3 steps. At first step, audio data is segmented into homogeneous regions in an unsupervised manner. After an ACTIVITY/NON-ACTIVITY decision is made for each region, ACTIVITY regions are classified as Speech/Non-spe...
A Low cost learning based sign language recognition system
Akış, Abdullah Hakan; Akar, Gözde; Department of Electrical and Electronics Engineering (2018)
Sign Language Recognition (SLR) is an active area of research due to its important role in Human Computer Interaction (HCI). The aim of this work is to automatically recognize hand gestures consisting of the movement of hand, arm and fingers. To achieve this, we studied two different approaches, namely feature based recognition and Convolutional Neural Networks (CNN) based recognition. The first approach is based on segmentation, feature extraction and classification whereas the second one is based on segme...
Finite-state sign language morphophonology
Sevinç, Ayça Müge; Bozşahin, Hüseyin Cem; Department of Cognitive Sciences (2015)
The aim of this thesis is to investigate the computational power required for processing sign language morphophonology. This dissertation focuses on the objective of reducing autosegmental representations and rules defined by three sign language phonology models, namely, Movement-Hold Model (Liddell & Johnson, 1989), Hand-Tier Model (Sandler, 1989, 1990), and Prosodic Model (Brentari, 1998), to finite state machinery. By adopting Autosegmental Phonology framework (Goldsmith, 1976), these models are capable ...
Hierarchical representations for visual object tracking by detection
Beşbınar, Beril; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2015)
Deep learning is the discipline of training computational models that are composed of multiple layers and these methods have improved the state of the art in many areas such as visual object detection, scene understanding or speech recognition. Rebirth of these fairly old computational models is usually related to the availability of large datasets, increase in the computational power of current hardware and more recently proposed unsupervised training methods that exploit the internal structure of very lar...
Citation Formats
B. Büyüksaraç, “Sign language recognition by image analysis /,” M.S. - Master of Science, Middle East Technical University, 2015.