Real - time stereo to multi-view video conversion

Çığla, Cevahir
A novel and efficient methodology is presented for the conversion of stereo to multi-view video in order to address the 3D content requirements for the next generation 3D-TVs and auto-stereoscopic multi-view displays. There are two main algorithmic blocks in such a conversion system; stereo matching and virtual view rendering that enable extraction of 3D information from stereo video and synthesis of inexistent virtual views, respectively. In the intermediate steps of these functional blocks, a novel edge-preserving filter is proposed that recursively constructs connected support regions for each pixel among color-wise similar neighboring pixels. The proposed recursive update structure eliminates pre-defined window dependency of the conventional approaches, providing complete content adaptibility with quite low computational complexity. Based on extensive tests, it is observed that the proposed filtering technique yields better or competitive results against some leading techniques in the literature. The proposed filter is mainly applied for stereo matching to aggregate cost functions and also handles occlusions that enable high quality disparity maps for the stereo pairs. Similar to box filter paradigm, this novel technique yields matching of arbitrary-shaped regions in constant time. Based on Middlebury benchmarking, the proposed technique is currently the best local matching technique in the literature in terms of both precision and complexity. Next, virtual view synthesis is conducted through depth image based rendering, in which reference color views of left and right pairs are warped to the desired virtual view using the estimated disparity maps. A feedback mechanism based on disparity error is introduced at this step to remove salient distortions for the sake of visual quality. Furthermore, the proposed edge-aware filter is re-utilized to assign proper texture for holes and occluded regions during view synthesis. Efficiency of the proposed scheme is validated by the real-time implementation on a special graphics card that enables parallel computing. Based on extensive experiments on stereo matching and virtual view rendering, proposed method yields fast execution, low memory requirement and high quality outputs with superior performance compared to most of the state-of-the-art techniques.


3D Face Reconstruction Using Stereo Images and Structured Light
OZTURK, Ahmet Oguz; Halıcı, Uğur; ULUSOY PARNAS, İLKAY; AKAGUNDUZ, Erdem (2008-04-22)
In this paper, the 3D face scanner that we developed using stereo cameras and structured light together is presented. Structured light having a pattern of vertical lines is used to create feature points and to match them easily. 3D point cloud obtained by stereo analysis is post processed to obtain the 3D model in obj format.
Planar 3d scene representations for depth compression
Özkalaycı, Burak Oğuz; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2014)
The recent invasion of stereoscopic 3D television technologies is expected to be followed by autostereoscopic and holographic technologies. Glasses-free multiple stereoscopic pair displaying capabilities of these technologies will advance the 3D experience. The prospective 3D format to create the multiple views for such displays is Multiview Video plus Depth (MVD) format based on the Depth Image Based Rendering (DIBR) techniques. The depth modality of the MVD format is an active research area whose main obj...
Stereo video broadcasting over DVB-H
Buğdaycı, Döne; Akar, Gözde; Department of Electrical and Electronics Engineering (2012)
This thesis proposes a complete framework of an end-to-end transmission of stereo video to mobile devices using DVB-H. Block diagram of the system is presented and operations conducted on the video at each layer are explained. Parameters and methodologies that may make a robust transmission possible are discussed. The transmission performance is analyzed in terms of error robustness under various coding methods, prediction structures, layering and protection strategies for different contents and channel con...
Photometric stereo considering highlights and shadows
Büyükatalay, Soner; Halıcı, Uğur; Birgül, Özlem; Department of Electrical and Electronics Engineering (2011)
Three dimensional (3D) shape reconstruction that aims to reconstruct 3D surface of objects using acquired images, is one of the main problems in computer vision. There are many applications of 3D shape reconstruction, from satellite imaging to material sciences, considering a continent on earth or microscopic surface properties of a material. One of these applications is the automated firearm identification that is an old, yet an unsolved problem in forensic science. Firearm evidence matching algorithms rel...
A complexity-utility framework for optimizing quality of experience for visual content in mobile devices
Önür, Özgür Deniz; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2012)
Subjective video quality and video decoding complexity are jointly optimized in order to determine the video encoding parameters that will result in the best Quality of Experience (QoE) for an end user watching a video clip on a mobile device. Subjective video quality is estimated by an objective criteria, video quality metric (VQM), and a method for predicting the video quality of a test sequence from the available training sequences with similar content characteristics is presented. Standardized spatial i...
Citation Formats
C. Çığla, “Real - time stereo to multi-view video conversion,” Ph.D. - Doctoral Program, Middle East Technical University, 2012.