Optical flow based video frame segmentation and segment classification

Akpınar, Samet
Video information retrieval is a field of multimedia research enabling us to extract desired semantic information from video data. In content-based video information retrieval, visual content obtained from video scenes is utilized. For developing methods to cope with content-based video information retrieval in terms of temporal concepts such as action, event, etc., representation of temporal information becomes critical. In this thesis, action detection is tackled based on a temporal video representation model. Herein, the visual feature - optical flow - is our basic construct used to formalize video parts as temporal information. In the proposed model, video action detection is considered over a pieced approach composed of two parts; Temporal video segment classification and temporal video segmentation. In the first part, weighted frame velocity concept is put forward and associated with the optical flow vectors. The associated representation is used in action based video segment classification. The second part contains a new temporal video segmentation methodology providing segment candidates to segment classification methods generally. The methodology brings an approach strengthening the pixel based cut detection methods with the motion based ones. Average motion vectors are presented based on the optical flow vectors and used in pixel matching. A binary cut classification is applied to the obtained representation enriched with a sliding window based approach. Proposed methods are applied to different data sets. Analysis of the results with the state of the art methods shows that proposed temporal representation models and concepts increased the segment and cut classification performances.
Citation Formats
S. Akpınar, “Optical flow based video frame segmentation and segment classification,” Ph.D. - Doctoral Program, Middle East Technical University, 2018.