Deep joint deinterlacing and denoising for single shot dual-ISO HDR reconstruction

Download
2019
Çoğalan, Uğur
HDR (High Dynamic Range) images have traditionally been obtained by merging multiple exposures each captured with a different exposure time. However, this approach entails longer capture times and necessitates deghosting if the captured scene contains moving objects. With the advent of modern camera sensors that can perform per-pixel exposure modulation, it is now possible to capture all of the required exposures within a single shot. The new challenge then becomes how to best combine different pixels with different exposure values into a single full-resolution and low-noise HDR image. In this thesis, we propose a joint multi-exposure frame deinterlacing and denoising algorithm powered by deep convolutional neural networks (DCNN). In our algorithm, we first train two DCNNs, with one tuned for reconstructing low exposures and the other for high exposures. Each DCNN takes the same mosaicked dual-ISO input image and outputs either the low exposure or high exposure depending on the type of the network. The resulting exposures can be demosaicked and converted to the desired target color space prior to HDR assembly. Our evaluations using computational metrics as well as visual comparisons indicate that the quality of our reconstructions significantly surpasses the state-of-the-art in this field.

Suggestions

Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction
Cogalan, Ugur; Akyüz, Ahmet Oğuz (Institute of Electrical and Electronics Engineers (IEEE), 2020-01-01)
HDR images have traditionally been obtained by merging multiple exposures each captured with a different exposure time. However, this approach entails longer capture times and necessitates deghosting if the captured scene contains moving objects. With the advent of modern camera sensors that can perform per-pixel exposure modulation, it is now possible to capture all of the required exposures within a single shot. The new challenge then becomes how to best combine different pixels with different exposure va...
An Evaluation of ghost removal algorithms for exposure fusion
Kutlu, Tuğser; Akar, Gözde; Department of Electrical and Electronics Engineering (2015)
In high dynamic range imaging (HDR), the goal is to capture a scene with a higher dynamic range than the camera capable of capturing with a single exposure. Similar to HDR, exposure fusion is a process that takes multiple images and combines them to create a single dynamically enhanced image by only keeping the properly exposed elements. When using multiple images, local motion of objects can influence the quality of the final image in such a way that local motion of objects causes a ghost artifact. In this...
Ghost Removal in High Dynamic Range Images
Khan, Erum Arif; Akyüz, Ahmet Oğuz; Reinhard, Erik (2006-10-11)
High dynamic range images may be created by capturing multiple images of a scene with varying exposures. Images created in this manner are prone to ghosting artifacts, which appear if there is movement in the scene at the time of capture. This paper describes a novel approach to removing ghosting artifacts from high dynamic range images, without the need for explicit object detection and motion estimation. Weights are computed iteratively and then applied to pixels to determine their contribution to the fin...
Noise reduction in high dynamic range imaging
Akyüz, Ahmet Oğuz (Elsevier BV, 2007-10-01)
A common method to create high dynamic range (HDR) images is to combine several different exposures of the same scene. In this approach, the use of higher ISO settings will reduce exposure times, and thereby the total capture time. This is advantageous in certain environments where it may help minimize ghosting artifacts. However, exposures taken at high sensitivity settings tend to be noisy, which is further amplified by the HDR creation algorithm. We present a robust and efficient technique to significant...
The State of the art in HDR deghosting and an objective HDR image deghosting quality metric
Tursun, Okan Tarhan; Akyüz, Ahmet Oğuz; Department of Computer Engineering (2016)
Despite the emergence of new HDR acquisition methods, the multiple exposure technique (MET) is still the most popular one. The application of MET on dynamic scenes is a challenging task due to the diversity of motion patterns and uncontrollable factors such as sensor noise, scene occlusion and performance concerns on some platforms with limited computational capability. Currently, there are already more than 50 deghosting algorithms proposed for artifact-free HDR imaging of dynamic scenes and it is expected...
Citation Formats
U. Çoğalan, “Deep joint deinterlacing and denoising for single shot dual-ISO HDR reconstruction,” M.S. - Master of Science, Middle East Technical University, 2019.