Multimodal concept detection in broadcast media: KavTan

SOYSAL, Medeni
Alatan, Abdullah Aydın
TEKİN, Mashar
ESEN, Ersin
Acar, Banu Oskay
Ozan, Ezgi Can
Ates, Tugrul K.
ATIL, Ilkay
Ozkan, Savas
Arabaci, Mehmet Ali
ÖNÜR, Duygu
Alatan, A. Aydin
Çiloğlu, Tolga
Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the presented framework was assessed objectively over a wide range of semantic concepts (5 high-level, 14 visual, 9 audio, 2 supplementary) by using a significant amount of precisely labeled ground truth data. KavTan System achieves successful high-level concept detection performance in unconstrained TV broadcast by efficiently utilizing multimodal information that is systematically extracted from both spatial and temporal extent of multimedia data.


Graph-based multilevel temporal video segmentation
Sakarya, Ufuk; TELATAR, ZİYA (Springer Science and Business Media LLC, 2008-11-01)
This paper presents a graph-based multilevel temporal video segmentation method. In each level of the segmentation, a weighted undirected graph structure is implemented. The graph is partitioned into clusters which represent the segments of a video. Three low-level features are used in the calculation of temporal segments' similarities: visual content, motion content and shot duration. Our strength factor approach contributes to the results by improving the efficiency of the proposed method. Experiments sho...
RELIEF-MM: effective modality weighting for multimedia information retrieval
Yilmaz, Turgay; Yazıcı, Adnan; Kitsuregawa, Masaru (Springer Science and Business Media LLC, 2014-07-01)
Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algori...
A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos
Hatirnaz, Eren; Sah, Melike; Direkoglu, Cem (Springer Science and Business Media LLC, 2020-02-20)
Monitoring continuously captured surveillance videos is a challenging and time consuming task. To assist this issue, a new framework is introduced that applies anomaly detection, semantic annotation and provides a concept-based search interface. In particular, novel optical flow based features are used for abnormal crowd behaviour detection. Then, processed surveillance videos are annotated using a new semantic metadata model based on multimedia standards using Semantic Web technologies. In this way, global...
A temporal neural network model for constructing connectionist expert system knowledge bases
Alpaslan, Ferda Nur (Elsevier BV, 1996-04-01)
This paper introduces a temporal feedforward neural network model that can be applied to a number of neural network application areas, including connectionist expert systems. The neural network model has a multi-layer structure, i.e. the number of layers is not limited. Also, the model has the flexibility of defining output nodes in any layer. This is especially important for connectionist expert system applications.
A systematic approach to the integration of overlapping partitions in service-oriented data grids
Sunercan, H. Kevser; Alpdemir, M. Nedim; Çiçekli, Fehime Nihan (Elsevier BV, 2011-06-01)
This paper aims to provide a service-oriented data integration solution over data Grids for cases where distributed data sources are partitioned with overlapping sections of various proportions. This is an interesting variation which combines both replicated and partitioned data within the same data management framework. Thus, the data management infrastructure has to deal with specific challenges regarding the identification, access and aggregation of partitioned data with varying proportions of overlappin...
Citation Formats
M. SOYSAL et al., “Multimodal concept detection in broadcast media: KavTan,” MULTIMEDIA TOOLS AND APPLICATIONS, pp. 2787–2832, 2014, Accessed: 00, 2020. [Online]. Available: