A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos

Date

2020-02-20

Author

Hatirnaz, Eren
Sah, Melike
Direkoglu, Cem

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

287
views

0
downloads

Monitoring continuously captured surveillance videos is a challenging and time consuming task. To assist this issue, a new framework is introduced that applies anomaly detection, semantic annotation and provides a concept-based search interface. In particular, novel optical flow based features are used for abnormal crowd behaviour detection. Then, processed surveillance videos are annotated using a new semantic metadata model based on multimedia standards using Semantic Web technologies. In this way, globally inter-operable metadata about abnormal crowd behaviours are generated. Finally, for the first time, based on crowd behaviours, a novel concept-based semantic search interface is proposed. In the proposed interface, along with search results (video segments), statistical data about crowd behaviours are also presented. With extensive user studies, it is demonstrated that the proposed concept-based semantic search interface enables efficient search and analysis of abnormal crowd behaviours. Although there are existing works to achieve (a) crowd anomaly detection, (b) semantic annotation and (c) semantic search interface, none of the existing works combine these three system components in a novel framework like the one proposed in this paper. In each system component, we introduce contributions to the field as well as use the Semantic Web technologies to combine and standardize output of different system components; output of the anomaly detection is automatically annotated with metadata and stored to a semantic database. When continuous surveillance videos are processed, only the semantic database is updated. Finally, the user interface queries the updated database for searching/analyzing surveillance videos without changing any coding. Thus, the framework supports re-usability. This paper explains and evaluates different components of the framework.

Subject Keywords

Media Technology, Computer Networks and Communications, Hardware and Architecture, Software

URI

https://hdl.handle.net/11511/67007

Journal

MULTIMEDIA TOOLS AND APPLICATIONS

DOI

https://doi.org/10.1007/s11042-020-08659-2

Collections

Engineering, Article

Suggestions

OpenMETU
Core

Semantic Annotation of Surveillance Videos for Abnormal Crowd Behaviour Search and Analysis Sah, Melike; Direkoglu, Cem (2017-09-01) Monitoring videos captured by surveillance cameras is a very difficult and time consuming task. There is a need for automated analysis using computer vision methods in order to recognize abnormal human behaviors and assist authorities. On the other hand, crowd (group of people) behavior analysis is a new direction of research, which can be utilized for automatic detection of panic in crowds. Once, videos are processed using computer vision technologies, another problem is how this data is indexed for search...
Multimodal concept detection in broadcast media: KavTan SOYSAL, Medeni; Alatan, Abdullah Aydın; TEKİN, Mashar; ESEN, Ersin; SARACOĞLU, Ahmet; Acar, Banu Oskay; Ozan, Ezgi Can; Ates, Tugrul K.; SEVİMLİ, Hakan; SEVİNÇ, Muge; ATIL, Ilkay; Ozkan, Savas; Arabaci, Mehmet Ali; TANKIZ, Seda; KARADENİZ, Talha; ÖNÜR, Duygu; SELÇUK, Sezin; Alatan, A. Aydin; Çiloğlu, Tolga (Springer Science and Business Media LLC, 2014-10-01) Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the p...
Efficient active rule processing in wireless multimedia sensor networks Oztarak, Hakan; Akkaya, Kemal; Yazıcı, Adnan; Sarisaray-Boluk, Pinar (Inderscience Publishers, 2016-01-01) Due to limited energy resources in wireless multimedia sensor networks (WMSNs), there is a need to perform data reduction and elimination over raw video data at the camera sensors before transmission. Nonetheless, this data reduction and elimination may create imprecision and uncertainty in the data, reducing the quality of decision making. In this paper, we propose a reactive mechanism for not only fusing uncertain data at the sink but also for automated processing of data using active rules, extending the...
A systematic approach to the integration of overlapping partitions in service-oriented data grids Sunercan, H. Kevser; Alpdemir, M. Nedim; Çiçekli, Fehime Nihan (Elsevier BV, 2011-06-01) This paper aims to provide a service-oriented data integration solution over data Grids for cases where distributed data sources are partitioned with overlapping sections of various proportions. This is an interesting variation which combines both replicated and partitioned data within the same data management framework. Thus, the data management infrastructure has to deal with specific challenges regarding the identification, access and aggregation of partitioned data with varying proportions of overlappin...
A temporal neural network model for constructing connectionist expert system knowledge bases Alpaslan, Ferda Nur (Elsevier BV, 1996-04-01) This paper introduces a temporal feedforward neural network model that can be applied to a number of neural network application areas, including connectionist expert systems. The neural network model has a multi-layer structure, i.e. the number of layers is not limited. Also, the model has the flexibility of defining output nodes in any layer. This is especially important for connectionist expert system applications.

Citation Formats

E. Hatirnaz, M. Sah, and C. Direkoglu, “A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos,” MULTIMEDIA TOOLS AND APPLICATIONS, pp. 0–0, 2020, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/67007.