EXPLORING DENSE DEPTH PREDICTIONS AS A SUPERVISION SOURCE FOR HUMAN POSE AND SHAPE ESTIMATION

Date

2024-7-16

Author

Karagoz, Batuhan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

43
views

106
downloads

Cite This

This thesis examines the effectiveness of dense depth information obtained by state-of-the-art depth estimation models in learning 3D human pose and shape (HPS) estimation from a monocular image. Collecting ground-truth data to supervise HPS estimation is costly and constrained to limited lab environments. Researchers have used priors based on body structure or kinematics, cues obtained from other vision tasks such as optical flow and segmentation, and self-supervised tasks to mitigate the scarcity of supervision. Despite its apparent potential in this context, monocular depth estimation has yet to be explored. We address this by first defining a dense mapping and alignment between the points on the surface of the human mesh and the points reconstructed from depth estimation. We then propose and extensively evaluate several loss functions. Firstly, we introduce DDA that combines dense mapping and alignment with a simple minimum squared distance loss. We extend it with DRT, a loss function to enforce the similarity transform between the target and the predicted body parts to be close to identity. Lastly, we propose MotionDRT that enforces motion consistency between depth and HPS predictions to train video HPS estimation models. Our experiments on Human3.6M and 3DPW datasets show that the proposed mapping, alignment, and loss calculation pipeline significantly improves HPS estimation performance over using only 2D keypoint supervision or 2D and 3D supervision.

Subject Keywords

Human Pose and Shape Estimation, Dense Depth Estimation, Dense Keypoint Estimation

URI

https://hdl.handle.net/11511/110614

Collections

Graduate School of Natural and Applied Sciences, Thesis