Creating a generic hand and finger gesture recognizer by using forearm muscle activity signals

2017
Demirel, Umut
Hand and finger gestures are one of the most natural ways of non-verbal communication. Apart from their daily use in different cultures, they are also widely used in human-computer interaction. There are a variety of applications using gestures as inputs such as sign language recognition, robot control, mobile phone control, medical device control and video game control. Advances in near-field wireless communications made it possible to design and deploy low-cost, inconspicuous control devices which can be used to detect certain predefined hand gestures for use in interaction. This thesis aims to investigate and develop a generic hand and finger gesture recognizer by processing forearm muscle activity signals from such a device which consists of eight electromyography (EMG) and Inertial Measurement Unit (IMU) sensors. Two main presuppositions of this thesis is that i) the muscle activity on the forearm is a spatiotemporally bandlimited circular signal and that can be sampled using a finite number of sensors, and ii) different gestures result in different but consistent patterns which are separable. The approach used in this thesis is based on the extraction of features by joint processing of signals obtained from a commercially available, low-cost EMG armband and classification of the gestures by simple and low-complexity artificial neural networks (ANNs). The dictionary of gestures were chosen from a canonical catalog of expressive gestures of classical orchestra conductors. Two experiments were carried out to assess the system performance: i) thirteen different hand and finger gestures and one rest gesture are performed 5 times in the same session by 10 different subjects, and ii) data was collected in an ecological study from a conductor during a practice session of a symphony orchestra. It was found that the proposed method achieved an average classification ac- curacy of 63.14% (maximum of 79.87%) when the data distribution for train, test and validate parts of ANN used in classification process is separated by sessions. An average classification accuracy of 96.09% (maximum of 98.8%) was achieved when data distribution is random. Lastly, random data distribution of the five different gestures and one rest gesture data collected in an ecological study from a conductor resulted in 96.9% accuracy. All results were obtained with session and subject dependent experiments.

Suggestions

A Low cost learning based sign language recognition system
Akış, Abdullah Hakan; Akar, Gözde; Department of Electrical and Electronics Engineering (2018)
Sign Language Recognition (SLR) is an active area of research due to its important role in Human Computer Interaction (HCI). The aim of this work is to automatically recognize hand gestures consisting of the movement of hand, arm and fingers. To achieve this, we studied two different approaches, namely feature based recognition and Convolutional Neural Networks (CNN) based recognition. The first approach is based on segmentation, feature extraction and classification whereas the second one is based on segme...
Fluid swarm formation control for hand gesture imitation by ellipse fitting
TİLKİ, Umut; Erkmen, Aydan Müşerref (Institution of Engineering and Technology (IET), 2015-03-19)
The main focus of this reported work is to imitate human hand gestures in the presence of the organ matching problem. Towards this aim, a case where the imitator is a fluidic system is adopted, whose dynamics are totally different from the imitatee; that is, a human performing different hand gestures. Proposed is a fluidics formation control of fluid particles where the formations result from the imitation of observed human hand gestures. Fluid body motion is modelled in a particle-based approach by using s...
Learning to Generate Unambiguous Spatial Referring Expressions for Real-World Environments
Dagan, Fethiye Irmak; Kalkan, Sinan; Leite, Iolanda (2019-01-01)
Referring to objects in a natural and unambiguous manner is crucial for effective human-robot interaction. Previous research on learning-based referring expressions has focused primarily on comprehension tasks, while generating referring expressions is still mostly limited to rule-based methods. In this work, we propose a two-stage approach that relies on deep learning for estimating spatial relations to describe an object naturally and unambiguously with a referring expression. We compare our method to the...
Effect of the vestibular system on search and fall behaviour of human postural sway
Cengiz, Berat Can; Gürses, Senih; Department of Engineering Sciences (2017)
In a person's daily life, upright stance is a very essential postural position for using hands, locomotion, communication, etc. These motivations probably help a baby to solve this complex motor task faster. A healthy individual is thought to maintain his/her upright posture mainly by the help of four senses: vestibular, proprioception, vision, and somatosensory. With a classical view, human upright posture control mechanism is approached as a control system with multi inputs almost single output (CoPx). Ho...
3D animation for hand preshaping
Güler, Mehmet Soner; Halıcı, Uğur; Department of Electrical and Electronics Engineering (2006)
The human hand is an essential part of human body, capable of making complex and expressive motions. Its complicated structure makes it a formidable challenge for animators to animate hand motions. Most computer graphics research on hand motion has focused on preshaping, preshaping and gestures with application to areas of human computer interaction and sign language. There are also a number of educational applications such as typing, playing of musical instruments etc. From a computer graphics standpoint, ...
Citation Formats
U. Demirel, “Creating a generic hand and finger gesture recognizer by using forearm muscle activity signals,” M.S. - Master of Science, Middle East Technical University, 2017.