Activity prediction from auto-captured lifelog images

Download

index.pdf

Date

2019

Author

Belli, Kader

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

222
views

126
downloads

The analysis of lifelogging has generated great interest among data scientists because large-scale, multidimensional and multimodal data are generated as a result of lifelogging activities. In this study, we use the NTCIR Lifelog dataset where daily lives of two users are monitored for a total of 90 days, and archived as a set of minute-based records consisting of details like semantic location, body measurements, listening history, and user activity. In addition, images which are captured automatically by cameras located at users' chests are available for each minute together with text annotations, which promotes the multimodal nature of the dataset. We train and evaluate several classification methods on the text and image data separately, and on their combination as well. Specifically, for text data, we encode the words using a one-hot encoding, and train SVM and MLP models on bag-of-words representations of minutes. For image data, we train two different convolutional neural networks (CNN) in two different ways: training from scratch and fine-tuning an ImageNet pre-trained model. Finally, we propose a multi-loss, combined CNN-MLP model which processes image and text data simultaneously, uses fusion methods to merge the two sub-models, and can handle missing input modalities. We also put effort into a contribution to the NTCIR LifeLog dataset by manually labeling 90,000 images into 16 activity classes

Subject Keywords

Event processing (Computer science)., Keywords: Lifelog, multimodal classification, machine learning, deep learning.

URI

http://etd.lib.metu.edu.tr/upload/12623472/index.pdf
https://hdl.handle.net/11511/43592

Collections

Graduate School of Natural and Applied Sciences, Thesis

Suggestions

OpenMETU
Core

Activity Learning from Lifelogging Images Belli, Kader; Akbaş, Emre; Yazıcı, Adnan (2019-01-01) The analytics of lifelogging has generated great interest for data scientists because big and multi-dimensional data are generated as a result of lifelogging activities. In this paper, the NTCIR Lifelog dataset is used to learn activities from an image point of view. Minute definitions are classified into activity classes using images and annotations, which serve as a basis for various classification techniques, namely SVMs and convolutional neural network structures (CNN), for learning activities. The perf...
PROGRESSIVE CLUSTERING OF MANIFOLD-MODELED DATA BASED ON TANGENT SPACE VARIATIONS Gokdogan, Gokhan; Vural, Elif (2017-09-28) An important research topic of the recent years has been to understand and analyze manifold-modeled data for clustering and classification applications. Most clustering methods developed for data of non-linear and low-dimensional structure are based on local linearity assumptions. However, clustering algorithms based on locally linear representations can tolerate difficult sampling conditions only to some extent, and may fail for scarcely sampled data manifolds or at high-curvature regions. In this paper, w...
Privacy-preserving horizontal federated learning methodology through a novel boosting-based federated random forest algorithm Gençtürk, Mert; Çiçekli, Fehime Nihan; Department of Computer Engineering (2023-1-04) In this thesis, a novel federated ensemble classification algorithm for horizontally partitioned data called Boosting-based Federated Random Forest (BOFRF) is proposed, which not only increases the predictive power of all participating sites, but also provides significantly high improvement on the predictive power of sites having unsuccessful local models. In this regard, a federated version of random forest, which is a well-known bagging algorithm, is implemented by adapting the idea of boosting to it. In ...
Binary Classification Performance Measures/Metrics: A Comprehensive Visualized Roadmap to Gain New Insights Canbek, Gurol; SAĞIROĞLU, Şeref; Taşkaya Temizel, Tuğba; Baykal, Nazife (2017-10-08) Binary classification is one of the most frequent studies in applied machine learning problems in various domains, from medicine to biology to meteorology to malware analysis. Many researchers use some performance metrics in their classification studies to report their success. However, the literature has shown a widespread confusion about the terminology and ignorance of the fundamental aspects behind metrics. This paper clarifies the confusing terminology, suggests formal rules to distinguish between meas...
Data mining analysis of economic indicators of countries Güngör, Erdem; Yozgatlıgil, Ceylan; Department of Statistics (2020-8) Data Mining is becoming a famous analysis day by day to reveal the hidden information within big data. In the study, we use data mining techniques on the economic indicators of the countries. The four data mining techniques are to be implemented on the dataset. Making homogenous groups of the countries whose economic characteristics are similar are obtained by the Clustering Algorithm. After the clustering algorithm is performed, we pass to Association Rule Data Mining to investigate the most exported produ...

Citation Formats

K. Belli, “Activity prediction from auto-captured lifelog images,” Thesis (M.S.) -- Graduate School of Natural and Applied Sciences. Computer Engineering., Middle East Technical University, 2019.