CROWD COUNTING, LOCALIZATION AND ANOMALY DETECTION WITH CONVLSTM BASED CNN USING SYNTHETIC IMAGES

2022-6-28
Coşkun, Muhammet Furkan
Crowd analysis related tasks have a wide range of real-life applications such as mass surveillance, safety monitoring, disaster management, traffic control and public area design. There are different novel works on crowd counting, localization and abnormal event detection tasks in the computer vision literature. In this study, we propose a ConvLSTM based Convolutional Neural Network to solve these three problems together in a single network. Our proposed model is a combination of P2P-Net architecture, which is a crowd counting and localization architecture, with a novel crowd anomaly detection module. P2P-Net architecture uses a VGG-16 network as the backbone and directly predicts the point proposals that represent the human heads. The proposed crowd anomaly head consists of an LSTM encoder with two LSTM cells and a fully convolutional anomaly binary classifier head. The proposed architecture has single backbone, multiple head structure and makes predictions for three different crowd tasks: crowd counting, crowd localization and crowd anomaly detection. We train the whole model in an end-to-end manner using synthetic data. Since counting and localization in crowd problem requires pixel-level annotations, real-world dataset creation process involves significant manual effort. Moreover, since an unified definition for crowd anomaly situation has not existed yet, there are different crowd anomaly datasets that have different anomaly definitions. For this reason, multiple datasets couldn't be used together to train a network. Moreover, none of them are big enough to train a decent ConvLSTM network. To overcome this data problem, we use synthetic datasets for network training. There is a synthetic dataset that fits our problem, called GTA-Events dataset. However, since the amount of data and scene variations is not enough in this dataset, we created a realistic synthetic dataset using the GTA-V game. Our novel synthetic dataset, named as METU Synthetic Crowd Dataset(METU-SCD), includes different scenarios, weather conditions and time intervals of the day. We train our proposed model with our novel synthetic dataset along with the GTA-Events dataset. Results show that the proposed model reaches over 90\% accuracy in crowd anomaly detection task and outperform optical-flow based methods. In crowd counting and localization tasks, it outperform the base model(P2P-Net) on synthetic data, whereas the real data results are somewhat behind than other methods. We, also, conduct an ablation study on the effect of counting and regression losses to anomaly performance. It is shown that training the network with a combination of counting, regression and anomaly losses brings increase in anomaly prediction score compared to the case that only anomaly loss is used.

Suggestions

IPBM: an energy efficient reliable interference-aware periodic broadcast messaging protocol for MANETs
ÜNLÜ, BERK; Ozceylan, Baver; Baykal, Buyurman (Springer Science and Business Media LLC, 2019-07-01)
Mobile ad-hoc networks (MANETs) have been widely employed in many fields including critical information delivery in open terrains as in tactical area, vehicular or disaster area network scenarios. To provide effective network maintenance for those MANETs, it is essential to adopt proper control communication methods, which provide reliable delivery of network information. However, it is difficult to provide control communication that meets the quality of service requirements due to the broadcasting of contr...
Text recognition and correction for automated data collection by mobile devices
Ozarslan, Suleyman; Eren, Pekin Erhan (2014-02-06)
Participatory sensing is an approach which allows mobile devices such as mobile phones to be used for data collection, analysis and sharing processes by individuals. Data collection is the first and most important part of a participatory sensing system, but it is time consuming for the participants. In this paper, we discuss automatic data collection approaches for reducing the time required for collection, and increasing the amount of collected data. In this context, we explore automated text recognition o...
Face Recognition Based on Embedding Learning
Karaman, Kaan; Koc, Aykut; Alatan, Abdullah Aydın (2018-09-11)
Face recognition is a key task of computer vision research that has been employed in various security and surveillance applications. Recently, the importance of this task has risen with the improvements in the quality of sensors of cameras, as well as with the increasing coverage of camera networks setup everywhere in the cities. Moreover, biometry-based technologies have been developed for the last three decades and have been available on many devices such as the mobile phones. The goal is to identify peop...
Wireless networking in tunnelling projects
Nielsen, Yasemin; Koseoglu, Ozan (Elsevier BV, 2007-05-01)
Emerging collaborative technologies and working methods often require tremendous engineering and organisational efforts for successful implementation of information and communication technologies (ICTs). According to a research conducted in Turkish construction industry, most of the problems that occur during the construction phase are due to lack of co-operation and communication between designers and contractors and lack of prompt expert decisions during on-site engineering. Similar to any major construct...
Multilevel Object Tracking in Wireless Multimedia Sensor Networks for Surveillance Applications Using Graph-Based Big Data
Kucukkececi, Cihan; Yazıcı, Adnan (Institute of Electrical and Electronics Engineers (IEEE), 2019-01-01)
Wireless Multimedia Sensor Networks (WMSN), for object tracking, have been used as an emerging technology in different application areas, such as health care, surveillance, and traffic control. In surveillance applications, sensor nodes produce data almost in real-time while tracking the objects in a critical area or monitoring border activities. The generated data is generally treated as big data and stored in NoSQL databases. In this paper, we present a new object tracking approach for surveillance applic...
Citation Formats
M. F. Coşkun, “CROWD COUNTING, LOCALIZATION AND ANOMALY DETECTION WITH CONVLSTM BASED CNN USING SYNTHETIC IMAGES,” M.S. - Master of Science, Middle East Technical University, 2022.