Multi-modal dialog scene detection using hidden Markov models for content-based multimedia indexing

2001-06-01
Alatan, Abdullah Aydın
AKANSU, ALİ N
WOLF, WAYNE
A class of audio-visual data (fiction entertainment: movies, TV series) is segmented into scenes, which contain dialogs, using a novel hidden Markov model-based (HMM) method. Each shot is classified using both audio track (via classification of speech, silence and music) and visual content (face and location information). The result of this shot-based classification is an audio-visual token to be used by the HMM state diagram to achieve scene analysis. After simulations with circular and left-to-right HMM topologies, it is observed that both are performing very good with multi-modal inputs. Moreover, for circular topology, the comparisons between different training and observation sets show that audio and face information together gives the most consistent results among different observation sets.

Citation Formats
A. A. Alatan, A. N. AKANSU, and W. WOLF, “Multi-modal dialog scene detection using hidden Markov models for content-based multimedia indexing,” MULTIMEDIA TOOLS AND APPLICATIONS, vol. 14, no. 2, pp. 137–151, 2001, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/36487.