Depth assisted object segmentation in multi-view video

Cigla, Cevahir
Alatan, Abdullah Aydın
In this work, a novel and unified approach for multi-view video (MVV) object segmentation is presented. In the first stage, a region-based graph-theoretic color segmentation algorithm is proposed, in which the popular Normalized Cuts segmentation method is improved with some modifications on its graph structure. Segmentation is obtained by recursive bi-partitioning of a weighted graph of an initial over-segmentation mask. The available segmentation mask is also utilized during dense depth map estimation step, based on a novel modified plane- and angle-sweeping strategy for each of these regions. Dense depth estimation is achieved by region-wise planarity assumption for the whole scene, in which depth models are estimated for sub-regions. Finally, the multi-view image segmentation algorithm is extended to object segmentation in MVV by the additional optical flow information. The required motion field is obtained via region-based matching that has consistent parameterization with color segmentation and dense depth map estimation algorithms. Experimental results indicate that proposed approach segments semantically meaningful objects in MVV with high precision.


Streaming Multiscale Deep Equilibrium Models
Ertenli, Can Ufuk; Akbaş, Emre; Cinbiş, Ramazan Gökberk (2022-1-01)
We present StreamDEQ, a method that infers frame-wise representations on videos with minimal per-frame computation. In contrast to conventional methods where compute time grows at least linearly with the network depth, we aim to update the representations in a continuous manner. For this purpose, we leverage the recently emerging implicit layer models, which infer the representation of an image by solving a fixed-point problem. Our main insight is to leverage the slowly changing nature of videos and use the...
Image segmentation with unified region and boundary characteristics within recursive shortest spanning tree
Esen, E.; Alp, Y. K. (2007-06-13)
The lack of boundary information in region based image segmentation algorithms resulted in many hybrid methods that integrate the complementary information sources of region and boundary, in order to increase the segmentation performance. In compliance with this trend, we propose a novel method to unify the region and boundary characteristics within the canonical Recursive Shortest Spanning Tree algorithm. The main idea is to incorporate the boundary information in the distance metric of RSST with minor cha...
Object recognition and segmentation via shape models
Altınoklu, Metin Burak; Ulusoy, İlkay; Tarı, Zehra Sibel; Department of Electrical and Electronics Engineering (2016)
In this thesis, the problem of object detection, recognition and segmentation in computer vision is addressed with shape based methods. An efficient object detection method based on a sparse skeleton has been proposed. The proposed method is an improved chamfer template matching method for recognition of articulated objects. Using a probabilistic graphical model structure, shape variation is represented in a skeletal shape model, where nodes correspond to parts consisting of lines and edges correspond to pa...
Cigla, Cevahir; Alatan, Abdullah Aydın (2009-01-01)
A method for estimating temporally and spatially consistent dense depth maps in multiple camera setups is presented which is important for reduction of perception artifacts in 3D displays. For this purpose, initially, depth estimation is performed for each camera with the piece-wise planarity assumption and Markov Random Field (MRF) based relaxation at each time instant independently. During the relaxation step, the consistency of depth maps for different cameras is also considered for the reliability of th...
Summarizing video: Content, features, and HMM topologies
Yasaroglu, Y; Alatan, Abdullah Aydın (2003-01-01)
An algorithm is proposed for automatic summarization of multimedia content by segmenting digital video into semantic scenes using HMMs. Various multi-modal low-level features are extracted to determine state transitions in HMMs for summarization. Advantage of using different model topologies and observation sets in order to segment different content types is emphasized and verified by simulations. Performance of the proposed algorithm is also compared with a deterministic scene segmentation method. A better...
Citation Formats
C. Cigla and A. A. Alatan, “Depth assisted object segmentation in multi-view video,” 2008, Accessed: 00, 2020. [Online]. Available: