An intelligent multimedia information system for multimodal content extraction and querying

Date

2018-01-01

Author

Yazıcı, Adnan
Yilmaz, Turgay
Sattari, Saeid
SERT, MUSTAFA
Gulen, Elvan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

320
views

0
downloads

This paper introduces an intelligent multimedia information system, which exploits machine learning and database technologies. The system extracts semantic contents of videos automatically by using the visual, auditory and textual modalities, then, stores the extracted contents in an appropriate format to retrieve them efficiently in subsequent requests for information. The semantic contents are extracted from these three modalities of data separately. Afterwards, the outputs from these modalities are fused to increase the accuracy of the object extraction process. The semantic contents that are extracted using the information fusion are stored in an intelligent and fuzzy object-oriented database system. In order to answer user queries efficiently, a multidimensional indexing mechanism that combines the extracted high-level semantic information with the low-level video features is developed. The proposed multimedia information system is implemented as a prototype and its performance is evaluated using news video datasets for answering content and concept-based queries considering all these modalities and their fused data. The performance results show that the developed multimedia information system is robust and scalable for large scale multimedia applications.

Subject Keywords

Multimedia applications, Prototype multimedia system, Multimedia platform, Video data, Semantic content extraction, Multimedia databases, Multimedia data fusion, Multimedia querying, Multimedia information retrieval

URI

https://hdl.handle.net/11511/49124

Journal

MULTIMEDIA TOOLS AND APPLICATIONS

DOI

https://doi.org/10.1007/s11042-017-4378-6

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

Flexible Content Extraction and Querying for Videos Demir, Utku; KOYUNCU, Murat; Yazıcı, Adnan; Yilmaz, Turgay; SERT, MUSTAFA (2011-10-28) In this study, a multimedia database system which includes a semantic content extractor, a high-dimensional index structure and an intelligent fuzzy object-oriented database component is proposed. The proposed system is realized by following a component-oriented approach. It supports different flexible query capabilities for the requirements of video users, which is the main focus of this paper. The query performance of the system (including automatic semantic content extraction) is tested and analyzed in t...
A hybrid swarm intelligence algorithm for simultaneous feature selection and clustering Geren, Hasan; Özdemirel, Nur Evin; Department of Industrial Engineering (2022-6-20) In this study, we address the feature selection and clustering problems by using a hybrid swarm intelligence approach. We assume that the number of clusters is known, clusters can be of any shape and have different densities, but there are no outliers or noise. The data set may have high dimensionality and redundant features. We propose a swarm intelligence algorithm, namely ACOVNS, which is a hybridization of Ant Colony Optimization (ACO) and Variable Neighborhood Search (VNS). We utilize the ACO mechanism...
An automated tool for information security management system Erkan, Ahmet; Arifoğlu, Ali; Department of Information Systems (2006) This thesis focuses on automation of processes of Information Security Management System. In accordance with two International Standards, ISO/IEC 27001:2005 and ISO/IEC 17799:2005, to automate the activities required for a documented ISMS as much as possible helps organizations. Some of the well known tools in this scope are analyzed and a comparative study on them including “InfoSec Toolkit”, which is developed for this purpose in the thesis scope, is given. “InfoSec Toolkit” is based on ISO/IEC 27001:2005...
Fusion of multimodal information for multimedia information retrieval Yılmaz, Turgay; Yazıcı, Adnan; Department of Computer Engineering (2014) An effective retrieval of multimedia data is based on its semantic content. In order to extract the semantic content, the nature of multimedia data should be analyzed carefully and the information contained should be used completely. Multimedia data usually has a complex structure containing multimodal information. Noise in the data, non-universality of any single modality, and performance upper bound of each modality make it hard to rely on a single modality. Thus, multimodal fusion is a practical approach...
A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data Karagoz, Gizem Nur; Yazıcı, Adnan; Dokeroglu, Tansel; Coşar, Ahmet (2020-06-01) There are few studies in the literature to address the multi-objective multi-label feature selection for the classification of video data using evolutionary algorithms. Selecting the most appropriate subset of features is a significant problem while maintaining/improving the accuracy of the prediction results. This study proposes a framework of parallel multi-objective Non-dominated Sorting Genetic Algorithms (NSGA-II) for exploring a Pareto set of non-dominated solutions. The subsets of non-dominated featu...

Citation Formats

A. Yazıcı, T. Yilmaz, S. Sattari, M. SERT, and E. Gulen, “An intelligent multimedia information system for multimodal content extraction and querying,” MULTIMEDIA TOOLS AND APPLICATIONS, pp. 2225–2260, 2018, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/49124.