Systematic evaluation of the effects of low-resolution and monochrome images on Instant Neural Graphic Primitives based 3D reconstruction

Date

2023-12

Author

Yılmaz, İlyas Eren

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

121
views

50
downloads

Cite This

Neural Radiance Fields (NeRF) have recently emerged as an alternative method to the classical 3D scene reconstruction approaches. Once trained on a set of 2D images taken from a complex scene, it can render novel views. Although many NeRF-based applications have successfully been demonstrated, there is no clear information on whether they can perform well in the case of limitations related to image resolution and colors. The main objective of this thesis is to conduct a systematic evaluation on the limitations of the state-of-the-art NeRF based 3D scene reconstruction methods employing low resolution and different color mode input images at various settings. By conducting these experiments, we intend to gain insight on whether current models can be used with low resolution images that are captured from the real-world systems such as small-scale quadrotors. Another goal is to determine the least amount of information required by the NeRF-based methods to perform successfully. In order to respond to these questions, a test environment is created to conduct experiments with Instant Neural Graphic Primitives (NGP) and NeRF together using input images having two color modes and several levels of low resolution. These images are generated from the Habitat Lab simulation, and camera parameters are estimated by COLMAP using these input image sets. The outputs of the experiments are reported in terms of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and learned perceptual image patch similarity (LPIPS) metrics. Results have shown that (i) higher resolution images create reconstructions of better quality, (ii) color mode affects each metric differently, and (iii) the quality of reconstruction is improved for low-resolution images when the intrinsics are estimated at their resolution and the extrinsics are estimated at a higher resolution. It is also observed that (iv) the best parameter configurations for Instant NGP might differ and should be fine-tuned for each scene scene separately, and (v) COLMAP reconstructions lose features and details and might contain holes while the Instant NGP might be a better candidate for reconstruction.

Subject Keywords

Reconstruction, Novel view synthesis, NeRF, SfM, COLMAP, Instant NGP

URI

https://hdl.handle.net/11511/107776

Collections

Graduate School of Natural and Applied Sciences, Thesis