Prioritized 3D scene reconstruction and rate-distortion efficient representation for video sequences

Download
2007
İmre, Evren
In this dissertation, a novel scheme performing 3D reconstruction of a scene from a 2D video sequence is presented. To this aim, first, the trajectories of the salient features in the scene are determined as a sequence of displacements via Kanade-Lukas-Tomasi tracker and Kalman filter. Then, a tentative camera trajectory with respect to a metric reference reconstruction is estimated. All frame pairs are ordered with respect to their amenability to 3D reconstruction by a metric that utilizes the baseline distances and the number of tracked correspondences between the frames. The ordered frame pairs are processed via a sequential structure-from-motion algorithm to estimate the sparse structure and camera matrices. The metric and the associated reconstruction algorithm are shown to outperform their counterparts in the literature via experiments. Finally, a mesh-based, rate-distortion efficient representation is constructed through a novel procedure driven by the error between a target image, and its prediction from a reference image and the current mesh. At each iteration, the triangular patch, whose projection on the predicted image has the largest error, is identified. Within this projected region and its correspondence on the reference frame, feature matches are extracted. The pair with the least conformance to the planar model is used to determine the vertex to be added to the mesh. The procedure is shown to outperform the dense depth-map representation in all tested cases, and the block motion vector representation, in scenes with large depth range, in rate-distortion sense.

Suggestions

3d reconstruction of underwater scenes from uncalibrated video sequences
Kırlı, Mustafa Yavuz; Ulusoy, İlkay; Department of Electrical and Electronics Engineering (2008)
The aim of this thesis is to reconstruct 3D representation of underwater scenes from uncalibrated video sequences. Underwater visualization is important for underwater Remotely Operated Vehicles and underwater is a complex structured environment because of inhomogeneous light absorption and light scattering by the environment. These factors make 3D reconstruction in underwater more challenging. The reconstruction consists of the following stages: Image enhancement, feature detection and matching, fundamenta...
Prioritized sequential 3D reconstruction in video sequences of dynamic scenes degrees
Imre, Evren; Alatan, Abdullah Aydın; Knorr, Sebastian; Sikora, Thomas (2006-04-19)
In this study, an algorithm is proposed to solve the multiframe structure from motion (MFSfM) problem for monocular video sequences in dynamic scenes. The algorithm uses the epipolar criterion to segment the features belonging to the independently moving objects. Once the features are segmented, the corresponding objects are reconstructed individually by using a sequential algorithm, which is also capable prioritizing the frame pairs with respect to their reliability and information content, thus achieving ...
Morphable 3D facial animation based on thin plate splines
Erdoğdu, Aysu; Ulusoy, İlkay; Department of Electrical and Electronics Engineering (2010)
The aim of this study is to present a novel three dimensional (3D) facial animation method for morphing emotions and facial expressions from one face model to another. For this purpose, smooth and realistic face models were animated with thin plate splines (TPS). Neutral face models were animated and compared with the actual expressive face models. Neutral and expressive face models were obtained from subjects via a 3D face scanner. The face models were preprocessed for pose and size normalization. Then mus...
3D indirect shape retrieval based on hand interaction
Irmak, Erdem Can; Sahillioğlu, Yusuf; Department of Game Technologies (2017)
In this thesis, a novel 3D indirect shape analysis method is presented which successfully retrieves 3D shapes based on the hand-object interaction. In the first part of the study, the human hand information is processed and trans- ferred to the virtual environment by Leap Motion Controller. Position and rotation of the hand, the angle of the finger joints are used for this part in our method. Also, in this approach, a new type of feature, which we call inter- action point, is introduced. These interaction p...
Prioritized sequential 3D reconstruction in video sequences with multiple motions
Imre, Evren; Knorr, Sebastian; Alatan, Abdullah Aydın; Sikora, Thomas (2006-10-11)
in this study, an algorithm is proposed to solve the multi-frame structure from motion (MFSfM) problem for monocular video sequences in dynamic scenes. The algorithm uses the epipolar criterion to segment the features belonging to independently moving objects. Once the features are segmented, corresponding objects are reconstructed individually by using a sequential algorithm, which is also capable of prioritizing the frame pairs with respect to their reliability and information content, thus achieving a fa...
Citation Formats
E. İmre, “Prioritized 3D scene reconstruction and rate-distortion efficient representation for video sequences,” Ph.D. - Doctoral Program, Middle East Technical University, 2007.