Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Open Access Guideline
Open Access Guideline
Postgraduate Thesis Guideline
Postgraduate Thesis Guideline
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
Hierarchical human activity recognition with fusion of audio and multiple inertial sensor modalities
Download
10451230.pdf
Date
2022-2-08
Author
Yılmaz, Tuğçe Alara
Metadata
Show full item record
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
.
Item Usage Stats
257
views
314
downloads
Cite This
People perform a wide variety of activities every day. Systems that can automatically distinguish these activities, i.e. human activity recognition models, have improved markedly, especially in the last decade. Deep learning is demonstrating increasingly promising outcomes in overcoming the problem of human activity detection as technology improves at a rapid pace. However, validating activity recognition in real-world situations is critical for practical solutions that work in natural contexts. Establishing systems that could achieve automatic activity recognition with real-life settings such as the devices that people use every day naturally and the environment they live in, might require lots of computational complexity. A lightweight neural network model is adopted for this purpose, one that could run swiftly even in the background process without taking up a lot of space when embedded into smartphones. Four inertial sensory data are represented in color coded image form and fused with three channelled audio data image representation to perform recognition task. The resulting fusion images allow rapid recognition performance because the size of the each image is so small. This thesis also underlines that audio sensor data, which require considerably bigger window sizes for identification on their own, improve automated recognition performance when used in conjunction with inertial sensors, even when divided into small window sizes to interact with other sensors simultaneously. In addition, this thesis provides a strategy that helps the computer better discern real-world behaviors by introducing activities, contexts, and placements in a hierarchical manner to perform accurate activity recognition. By merging auditory images with inertial color coded images, representing multiple activity pairs with hierarchical groups according to activity-context-placement, and employing a lightweight model, high accurate recognition performance score competitive to state-of-art, nearly 91\% success rate is achieved. We believe that this research could be classified as a "quality of experience" because it presents a lightweight model that could be used to predict behavior of individual by data collected from devices such as smartphone and smartwatch that everyone uses each day naturally.
Subject Keywords
Human activity recognition
,
Deep learning
,
Inertial sensor
,
Audio sensor
URI
https://hdl.handle.net/11511/96320
Collections
Graduate School of Natural and Applied Sciences, Thesis
Suggestions
OpenMETU
Core
Implementation of a Generic Framework on Crowd Simulation: A New Environment to Model Crowd Behavior and Design Video Games
Yücel, Furkan; Sürer, Elif (2020-12-01)
Crowd behavior is the collective act and gathering of a group of individuals to achieve a shared purpose. Swarm intelligence-based optimization algorithms are usually used to solve complex problems for crowd behavior. Crowd simulations are often used for the analyses that require precision in different domains such as complex structural analysis, image recognition, creating nature-inspired non-player character movements in video games, and more. In this study, a generic crowd simulation framework that can b...
A Workflow Based Mobile Guidance Framework for Managing Personal Activities
TÜYSÜZ, GÖKHAN; AVENOĞLU, BİLGİN; Eren, Pekin Erhan (2013-09-27)
In daily life, people have to perform a large number of activities typically in a limited amount of time. Accordingly, they may benefit from help and guidance provided by support systems in order to accomplish these activities accurately and in the correct order. In order to satisfy such needs we develop a software framework which also incorporates a mobile application. Within the framework, workflows are used for modeling user activities due to their successful structuring and verification capabilities. Ho...
Volunteers motivation and sense of community in a sports event
Yıldız, Alper; Koçak, Mehmet Settar; Yıldırım, Sinan; Devrilmez, Erhan (null; 2016-11-25)
Volunteers are indispensable for sports events. Success of the event is related with their well management and coordination. It is expected that keeping volunteers’ motivation and sense of community levels high will affect both volunteers and event positively. Hence, aim of this research was investigating the perceived senses of community and motivations of volunteers, and examining correlations between sense of community and motivation of sports event volunteers. Four hundred seventy-seven of whom voluntee...
A Workflow-based mobile guidance framework for managing personal activities
Tüysüz, Gökhan; Eren, Pekin Erhan; Department of Information Systems (2013)
In daily life, people have to perform a large number of activities typically in a limited amount of time. Thus, they may need help and guidance provided by support systems in order to accomplish these activities accurately and in the correct order. Accordingly, in this study, we propose a software framework based on workflows and supported by a mobile application to assist users in pervasive environments for managing their personal activities. Pervasive computing enables to ease and automate the execution o...
MULTI-SENSORY NATURE OF USER-PRODUCT INTERACTIONS: A STUDY ON SMALL HOUSEHOLD APPLIANCES
Coşkun, Merve; Şener Pedgley, Bahar (2018-07-01)
People are equipped with sensory systems that enable them to communicate with their surroundings and products. User-product interaction has a multi-sensory nature that includes vision, touch, kinesthetic, audition, smell and taste. Each sensory system is simultaneously active during an interaction. However, the role of other sensory qualities besides visual in product interactions and their contribution to the product experience are often overlooked. By taking multisensory nature of interaction into conside...
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
T. A. Yılmaz, “Hierarchical human activity recognition with fusion of audio and multiple inertial sensor modalities,” M.S. - Master of Science, Middle East Technical University, 2022.