Spatial synchronization of audiovisual objects by 3D audio object coding

2010-12-01
Günel Kılıç, Banu
Kondoz, Ahmet M.
Free viewpoint video enables the visualisation of a scene from arbitrary viewpoints and directions. However, this flexibility in video rendering provides a challenge in 3D media for achieving spatial synchronicity between the audio and video objects. When the viewpoint is changed, its effect on the perceived audio scene should be considered to avoid mismatches in the perceived positions of audiovisual objects. Spatial audio coding with such flexibility requires decomposing the sound scene into audio objects initially, and then synthesizing the new scene according to the geometric relations between the A/V capturing setup, selected viewpoint and the rendering system. This paper proposes a free viewpoint audio coding framework for 3D media systems utilising multiview cameras and a microphone array. A real-time source separation technique is used for object decomposition followed by spatial audio coding. Binaural, multichannel sound systems and wave field synthesis systems are addressed. Subjective test results shows that the method achieves spatial synchronicity for various viewpoints consistently, which is not possible by conventional recording techniques.

Suggestions

Temporal watermarking of digital video
Koz, A; Alatan, Abdullah Aydın (2003-04-11)
A video watermarking method is presented, based on the temporal sensitivity of Human Visual System (HVS). The method exploits the temporal contrast thresholds of HVS to determine the spatio-temporal locations, where the watermark should be embedded, and the maximum strength of watermark, which still gives imperceptible distortion after watermark insertion. The robustness results indicate that the proposed scheme survives video distortions, such as additive Gaussian noise, ITU H.263+ coding at medium bit rat...
Depth assisted object segmentation in multi-view video
Cigla, Cevahir; Alatan, Abdullah Aydın (2008-01-01)
In this work, a novel and unified approach for multi-view video (MVV) object segmentation is presented. In the first stage, a region-based graph-theoretic color segmentation algorithm is proposed, in which the popular Normalized Cuts segmentation method is improved with some modifications on its graph structure. Segmentation is obtained by recursive bi-partitioning of a weighted graph of an initial over-segmentation mask. The available segmentation mask is also utilized during dense depth map estimation ste...
Oblivious video watermaking using temporal sensitivity of HVS
Koz, A; Alatan, Abdullah Aydın (2004-04-30)
An oblivious video watermarking method is presented based on the temporal sensitivity of Human Visual System (HVS). The method exploits the temporal contrast thresholds of HVS to determine the maximum strength of watermark, which still gives imperceptible distortion after watermark insertion. Compared to other approaches in the literature, the method guarantees to avoid flickering problem in the watermarked video and gives better robustness results to video distortions, such as additive Gaussian noise, H.26...
Spatial Correlation in Single-Carrier Massive MIMO Systems
Beigiparast, Nader; Güvensen, Gökhan Muzaffer; Ayanoglu, Ender (2020-02-02)
© 2020 IEEE.We present the analysis of a single-carrier massive MIMO system for the frequency selective Gaussian multi-user channel, in both uplink and downlink directions. We develop expressions for the achievable sum rate when there is spatial correlation among antennas at the base station. It is known that the channel matched filter precoder (CMFP) performs the best in a spatially uncorrelated downlink channel. However, we show that, in a spatially correlated downlink channel with two different correlati...
Visual quality assessment for stereoscopic video sequences
Sarıkan, Selim Sefa; Akar, Gözde; Department of Electrical and Electronics Engineering (2011)
The aim of this study is to understand the effect of different depth levels on the overall 3D quality and develop an objective video quality metric for stereoscopic video sequences. Proposed method is designed to be used in video coding stages to improve overall 3D video quality. This study includes both objective and subjective evaluation. Test sequences with different coding schemes are used. Computer simulation results show that overall quality has a strong correlation with the quality of the background,...
Citation Formats
B. Günel Kılıç and A. M. Kondoz, “Spatial synchronization of audiovisual objects by 3D audio object coding,” 2010, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/30257.