Human presence detection in emergency situations using deep learning based audio-visual systems

Download

10495065.pdf

Date

2022-8-24

Author

Geneci, İzlen

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

225
views

89
downloads

The significance of emergency event detection in surveillance systems has drawn the attention of researchers in recent years. Existing methods mostly depend on visual data to identify any abnormal events since only visual sensors are frequently put in public settings. On the other hand, in an emergency, sound information may be exploited. When eyesight is occluded, audio waves can penetrate to some extent. Applications for visual analysis may be helpful when there is noise in the audio and the scene is congested. Thus, the shift from single-modality to multimodality learning has become crucial given the recent rapid growth of deep learning. Both the audio analysis and the visual analysis were performed separately. In audio-based analysis, audio was transformed into samples using sliding window technique to capture the brief window of a target audio class. Therefore, in a real-time operating system, emergency circumstances can be recognized when the target sound happens briefly. For human sound classes of "Speech", "Scream", "Cry", the minimum sliding window sizes were 0.25 s, 1 s and 0.30 s, respectively. In visual analysis, face detection was conducted along with facial alignment using five facial landmarks. The AP for face detection was 77% on WIDER Face dataset (IoU=0.5). Using the detected faces, facial expression recognition (FER) was performed as well as age and gender estimations by employing an attention-based method. For seven basic emotions, 64.14% accuracy was achieved on AffectNet dataset. The combination of these audio and visual-based systems eliminates the limitations of perceptual tasks in both modalities.

Subject Keywords

Audio Event Detection, Face Detection, Facial Expression Recognition

URI

https://hdl.handle.net/11511/99397

Collections

Graduate School of Informatics, Thesis

Suggestions

OpenMETU
Core

A FRAMEWORK FOR DETECTING COMPLEX EVENTS IN SURVEILLANCE VIDEOS Onal, Itir; Kardas, Karani; Rezaeitabar, Yousef; Bayram, Ulya; Bal, Murat; Ulusoy, İlkay; Cicekli, Nihan Kesim (2013-07-19) This paper presents a framework for detecting complex events in surveillance videos. Moving objects in the foreground are detected in the object detection component of the system. Whether these foregrounds are human or not is decided in the object recognition component. Then each detected object is tracked and labeled in the object tracking component, in which true labeling of objects in the occlusion situation is also provided. The extracted information is fed to the event detection component. Rule based e...
Automated Moving Object Classification in Wireless Multimedia Sensor Networks Civelek, Muhsin; Yazıcı, Adnan (2017-02-15) The use of wireless multimedia sensor networks (WMSNs) for surveillance applications has attracted the interest of many researchers. As with traditional sensor networks, it is easy to deploy and operate WMSNs. With inclusion of multimedia devices in wireless sensor networks, it is possible to provide data to users that is more meaningful than that provided by scalar sensor-based systems alone; however, producing, storing, processing, analyzing, and transmitting multimedia data in sensor networks requires co...
Airport runway detection in satellite images by Adaboost Learning ZÖNGÜR, Ugur; Halıcı, Uğur; AYTEKİN, Orsan; Ulusoy, İlkay (2009-09-03) Advances in hardware and pattern recognition techniques, along with the widespread utilization of remote sensing satellites, have urged the development of automatic target detection systems in satellite images. Automatic detection of airports is particularly essential, due to the strategic importance of these targets. In this paper, a runway detection method using a segmentation process based on textural properties is proposed for the detection of airport runways, which is the most distinguishing element of...
Event Detection by Change Tracking on Community Structure of Temporal Networks Aktunc, Riza; Toroslu, İsmail Hakkı; Karagöz, Pınar (2018-08-31) Event detection is a popular research problem, aiming to detect events from online data sources with least possible delay. Most of the previous work focus on analyzing textual content such as social media postings to detect happenings. In this work, we consider event detection as a change detection problem in network structure, and propose a method that detects change in community structure extracted from communication network. We study three versions of the method based on different change models. Experime...
Pedestrian zone anomaly detection by non-parametric temporal modelling Gündüz, Ayşe Elvan; Taşkaya Temizel, Tuğba; Temizel, Alptekin (2014-08-29) With the increasing focus on safety and security in public areas, anomaly detection in video surveillance systems has become increasingly more important. In this paper, we describe a method that models the temporal behavior and detects behavioral anomalies in the scene using probabilistic graphical models. The Coupled Hidden Markov Model (CHMM) method that we use shows that sparse features obtained via feature detection and description algorithms are suitable for modeling the temporal behavior patterns and ...

Citation Formats

İ. Geneci, “Human presence detection in emergency situations using deep learning based audio-visual systems,” M.S. - Master of Science, Middle East Technical University, 2022.