Video segmentation based on audio feature extraction

Atar, Neriman
In this study, an automatic video segmentation and classification system based on audio features has been presented. Video sequences are classified such as videos with “speech”, “music”, “crowd” and “silence”. The segments that do not belong to these regions are left as “unclassified”. For the silence segment detection, a simple threshold comparison method has been done on the short time energy feature of the embedded audio sequence. For the “speech”, “music” and “crowd” segment detection a multiclass classification scheme has been applied. For this purpose, three audio feature set have been formed, one of them is purely MPEG-7 audio features, other is the audio features that is used in [31] the last one is the combination of these two feature sets. For choosing the best feature a histogram comparison method has been used. Audio segmentation system was trained and tested with these feature sets. The evaluation results show that the Feature Set 3 that is the combination of other two feature sets gives better performance for the audio classification system. The output of the classification system is an XML file which contains MPEG-7 audio segment descriptors for the video sequence. An application scenario is given by combining the audio segmentation results with visual analysis results for getting audio-visual video segments.


Video Content Analysis Using Dominant Sets
Sakarya, Ufuk; TELATAR, ZİYA (2009-04-11)
In this paper, a graph-based method for video content analysis is proposed The characteristics of the detected shots are investigated for news, commercial, animated cartoon, basketball and documentary videos and experimental studies are realized on these videos. The maximum clique on the weighted and undirected graph, which is constructed according to visual content, is tried being detected. it is inferred that specially in news and commercials, the proposed method can be used for temporal video segmentation.
Video Content Analysis Method for Audiovisual Quality Assessment
Konuk, Baris; Zerman, Emin; NUR YILMAZ, GÖKÇE; Akar, Gözde (2016-06-08)
In this study a novel, spatio-temporal characteristics based video content analysis method is presented. The proposed method has been evaluated on different video quality assessment databases, which include videos with different characteristics and distortion types. Test results obtained on different databases demonstrate the robustness and accuracy of the proposed content analysis method. Moreover, this analysis method is employed in order to examine the performance improvement in audiovisual quality asses...
Selection and fusion of multiple stereo algorithms for accurate disparity segmentation
Bilgin, Arda; Ulusoy, İlkay; Department of Electrical and Electronics Engineering (2008)
Fusion of multiple stereo algorithms is performed in order to obtain accurate disparity segmentation. Reliable disparity map of real-time stereo images is estimated and disparity segmentation is performed for object detection purpose. First, stereo algorithms which have high performance in real-time applications are chosen among the algorithms in the literature and three of them are implemented. Then, the results of these algorithms are fused to gain better performance in disparity estimation. In fusion pro...
Video stabilization: digital and mechanical approaches
Bayrak, Serhat; Ulusoy, İlkay; Department of Electrical and Electronics Engineering (2008)
General video stabilization techniques which are digital, mechanical and optical are discussed. Under the concept of video stabilization, various digital motion estimation and motion correction algorithms are implemented. For motion estimation, in addition to digital approach, a mechanical approach is implemented also. Then all implemented motion estimation and motion correction algorithms are compared with respect to their computational times and accuracies over various videos. For small amount of jitter, ...
Dense depth map estimation for object segmentation in multi-view video
Çığla, Cevahir; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2007)
In this thesis, novel approaches for dense depth field estimation and object segmentation from mono, stereo and multiple views are presented. In the first stage, a novel graph-theoretic color segmentation algorithm is proposed, in which the popular Normalized Cuts 59H[6] segmentation algorithm is improved with some modifications on its graph structure. Segmentation is obtained by the recursive partitioning of the weighted graph. The simulation results for the comparison of the proposed segmentation scheme w...
Citation Formats
N. Atar, “Video segmentation based on audio feature extraction,” M.S. - Master of Science, Middle East Technical University, 2009.