Object Segmentation in Multi-view Video via Color, Depth and Motion Cues

Cigla, Cevahir
Alatan, Abdullah Aydın
In the light of dense depth map estimation, motion estimation and object segmentation, the research on multi-view video (MVV) content has becoming increasingly popular due to its wide application areas in the near future. In this work, object segmentation problem is studied by additional cues due to depth and motion fields. Segmentation is achieved by modeling images as graphical models and performing popular Normalized Cuts method with some modifications. In the graphical models, each node is represented by a group of pixels, instead of individual pixels, which are obtained as a result of over-segmentation of the images. These over-segmented regions are also utilized in the dense depth map estimation step; in which 3D planar models are assigned for each of these sub-regions. Moreover, optical flow is estimated based on affine motion assumption for these regions. The links of the graphical models are weighted according to the depth, motion and color similarities of the pixel groups due to these regions. Once the links are obtained, segmentation is achieved by recursively bi-partitioning the graph via removing the weak links. Experiments indicate that the proposed framework achieves precise segmentation results for MVV sequences.


Image segmentation with unified region and boundary characteristics within recursive shortest spanning tree
Esen, E.; Alp, Y. K. (2007-06-13)
The lack of boundary information in region based image segmentation algorithms resulted in many hybrid methods that integrate the complementary information sources of region and boundary, in order to increase the segmentation performance. In compliance with this trend, we propose a novel method to unify the region and boundary characteristics within the canonical Recursive Shortest Spanning Tree algorithm. The main idea is to incorporate the boundary information in the distance metric of RSST with minor cha...
Alatan, Abdullah Aydın (1994-01-01)
3-D motion estimation from a video sequence remains a challenging problem. Modelling the local interactions between the 3-D motion parameters is possible by using Gibbs random fields. An energy function which gives the joint probability distribution of the motion vectors, is constructed. The most probable motion vector set is found by maximizing the probability, represented by this distribution. Since the 3-D motion estimation problem is ill-posed, the regularization is achieved by an initial rigidity assum...
Recursive shortest spanning tree algorithms for image segmentation
Bayramoglu, NY; Bazlamaçcı, Cüneyt Fehmi (2005-11-24)
Image segmentation has an important role in image processing and the speed of the segmentation algorithm may become a drawback for some applications. This study analyzes the run time performances of some variations of the Recursive Shortest Spanning Tree Algorithm (RSST) and proposes simple but effective modifications on these algorithms to improve their speeds. In addition, the effect of link weight cost function on the run time performance and the segmentation quality is examined. For further improvement ...
Sarikan, Selim S.; Olgun, Ramazan F.; Akar, Gözde (2011-09-09)
This paper presents a new quality evaluation model for stereoscopic videos using depth map segmentation. This study includes both objective and subjective evaluation. The goal of this study is to understand the effect of different depth levels on the overall 3D quality. Test sequences with different coding schemes are used. The results show that overall quality has a strong correlation with the quality of the background, where disparity is smaller relative to the foreground. The results also showed that con...
3-D motion estimation of rigid objects for video coding applications using an improved iterative version of the E-matrix method
Alatan, Abdullah Aydın (1998-02-01)
As an alternative to current two-dimensional (2-D) motion models, a robust three-dimensional (3-D) motion estimation method is proposed to be utilized in object-based video coding applications, Since the popular E-matrix method is well known for its susceptibility to input errors, a performance indicator, which tests the validity of the estimated 3-D motion parameters both explicitly and implicitly, is defined. This indicator is utilized within the RANSAC method to obtain a robust set of 2-D motion correspo...
Citation Formats
C. Cigla and A. A. Alatan, “Object Segmentation in Multi-view Video via Color, Depth and Motion Cues,” 2009, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/47143.