A Method for isolated sign recognition with KINECT

Download
index.pdf
2014
Işıklıgil, Emre
Show full item record
181
views
67
downloads
Although there are various studies on sign language recognition (SLR), most of them use accessories like coloured gloves and accelerometers for data acquisition or require complex environmental setup to operate. In my thesis, I will use only Microsoft Kinect sensor for acquiring data for SLR. Kinect lets us obtain 3D positions of the body joints in real time without the help of any other device. After an isolated sign is captured, paths of the discriminative body joints are extracted. Then, a vector consisting of the extracted paths, called Sign Graph, is created to describe the isolated sign. To be able to compare two sign graphs, as the distance metric, I propose using the average warping distance of the joint paths that the sign graphs include. Dynamic Time Warping is used for effective calculation of the warping distance. Once a distance measure is defined between Sign Graphs, they are classified using k Nearest Neighbours algorithm. The proposed method performed better than the state of the art and achieved recognition rate of 59.3% in signer-independent experiments and 91.0% in signer-dependent experiments with a dataset consisting of 40 signs obtained from 13 different signers.
Pattern recognition systems., Sign language, Image processing, Computer graphics.
http://etd.lib.metu.edu.tr/upload/12617888/index.pdf
https://hdl.handle.net/11511/24013
Graduate School of Natural and Applied Sciences, Thesis

Suggestions

New method for the fusion of complementary information from infrared and visual images for object detection
Ulusoy, İlkay (Institution of Engineering and Technology (IET), 2011-02-01)
Visual and infrared cameras have complementary properties and using them together may increase the performance of object detection applications. Although the fusion of visual and infrared information results in a better recall rate than using only one of those domains, there is always a decrease in the precision rate whereas the infrared domain on its own always has higher precision. Thus, the fusion of these domains is meaningful only for a better recall rate, which means that more foreground pixels are de...
A Low cost learning based sign language recognition system
Akış, Abdullah Hakan; Akar, Gözde; Department of Electrical and Electronics Engineering (2018)
Sign Language Recognition (SLR) is an active area of research due to its important role in Human Computer Interaction (HCI). The aim of this work is to automatically recognize hand gestures consisting of the movement of hand, arm and fingers. To achieve this, we studied two different approaches, namely feature based recognition and Convolutional Neural Networks (CNN) based recognition. The first approach is based on segmentation, feature extraction and classification whereas the second one is based on segme...
Using multi-modal 3D contours and their relations for vision and robotics
BAŞESKİ, Emre; Pugeault, Nicolas; Kalkan, Sinan; BODENHAGEN, Leon; Piater, Justus H.; KRÜGER, Norbert (Elsevier BV, 2010-11-01)
In this work, we make use of 3D contours and relations between them (namely, coplanarity, cocolority, distance and angle) for four different applications in the area of computer vision and vision-based robotics. Our multi-modal contour representation covers both geometric and appearance information. We show the potential of reasoning with global entities in the context of visual scene analysis for driver assistance, depth prediction, robotic grasping and grasp learning. We argue that, such 3D global reasoni...
Camera electronics and image enhancement software for infrared detector arrays
Küçükkömürler, Alper; Akın, Tayfun; Department of Environmental Engineering (2012)
This thesis aims to design and develop camera electronics and image enhancement software for infrared detector arrays. It first discusses the camera electronics suitable for infrared detector arrays, then it concentrates on image enhancement software that are implemented including defective pixel correction, contrast enhancement, noise reduction and pseudo coloring. After that, testing and results of the implemented algorithms were presented. Camera electronics and circuit operation frequency are selected c...
A multimodal approach for individual tracking of people and their belongings
Beyan, Çiğdem; Temizel, Alptekin (2015-04-01)
In this study, a fully automatic surveillance system for indoor environments which is capable of tracking multiple objects using both visible and thermal band images is proposed. These two modalities are fused to track people and the objects they carry separately using their heat signatures and the owners of the belongings are determined. Fusion of complementary information from different modalities (for example, thermal images are not affected by shadows and there is no thermal reflection or halo effect in...
Citation Formats
E. Işıklıgil, “A Method for isolated sign recognition with KINECT,” M.S. - Master of Science, Middle East Technical University, 2014.
METU IIS - Integrated Information Service open@metu.edu.tr