Deep generation of 3D articulated models and animations from 2D stick figures

Download

1-s2.0-S0097849322001820-main.pdf

Date

2022-12-01

Author

Akman, Alican
Sahillioğlu, Yusuf
Sezgin, T. Metin

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

103
views

99
downloads

Generating 3D models from 2D images or sketches is a widely studied important problem in computer graphics. We describe the first method to generate a 3D human model from a single sketched stick figure. In contrast to the existing human modeling techniques, our method does not require a statistical body shape model. We exploit Variational Autoencoders to develop a novel framework capable of transitioning from a simple 2D stick figure sketch, to a corresponding 3D human model. Our network learns the mapping between the input sketch and the output 3D model. Furthermore, our model learns the embedding space around these models. We demonstrate that our network can generate not only 3D models, but also 3D animations through interpolation and extrapolation in the learned embedding space. In addition to 3D human models, we produce 3D horse models in order to show the generalization ability of our framework. Extensive experiments show that our model learns to generate compatible 3D models and animations with 2D sketches. (C) 2022 The Author(s). Published by Elsevier Ltd.

Subject Keywords

Computer graphics, 3D model generation, Deep learning, Sketch-based shape modeling, SHAPES

URI

https://hdl.handle.net/11511/101611

Journal

COMPUTERS & GRAPHICS-UK

DOI

https://doi.org/10.1016/j.cag.2022.10.004

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

Generation of 3D human models and animations using simple sketches Akman, Alican; Sahillioğlu, Yusuf; Sezgin, T. Metin (2020-01-01) Generating 3D models from 2D images or sketches is a widely studied important problem in computer graphics. We describe the first method to generate a 3D human model from a single sketched stick figure. In contrast to the existing human modeling techniques, our method requires neither a statistical body shape model nor a rigged 3D character model. We exploit Variational Autoencoders to develop a novel framework capable of transitioning from a simple 2D stick figure sketch, to a corresponding 3D human model....
End-to-end learned image compression with conditional latent space modeling for entropy coding Yesilyurt, Aziz Berkay; Kamışlı, Fatih (2021-01-24) The use of neural networks in image compression enables transforms and probability models for entropy coding which can process images based on much more complex models than the simple Gauss-Markov models in traditional compression methods. All at the expense of higher computational complexity. In the neural-network based image compression literature, various methods to model the dependencies in the transform domain/latent space are proposed. This work uses an alternative method to exploit the dependencies o...
3D face reconstruction using stereo images and structured light Öztürk, Ahmet Oğuz; Halıcı, Uğur; Department of Electrical and Electronics Engineering (2007) Nowadays, 3D modelling of objects from multiple images is a topic that has gained great recognition and is widely used in various fields. Recently, lots of progress has been made in identification of people using 3D face models, which are usually reconstructed from multiple face images. In this thesis, a system including stereo cameras and structured light is built for the purpose of 3D modelling. The system outputs are 3D shapes of the face and also the texture information registered to this shape. Althoug...
3D face modeling using multiple images BUYUKATALAY, SONER; Halıcı, Uğur; AKAGUNDUZ, ERDEM; ULUSOY PARNAS, İLKAY (2006-04-19) 3D face modeling based on real images is one of the important subject of Computer Vision that is studied recently. In this paper the study that eve contucted in our Computer Vision and Intelligent Systems Research Laboratory on 3D face model generation using uncalibrated multiple still images is explained.
3D synthetic human face modelling tool based on T-spline surfaces Aydoğan, Ali; Ulusoy, İlkay; Department of Electrical and Electronics Engineering (2007) In this thesis work, a 3D Synthetic Human Face Modelling Software is implemented using C++ and OpenGL. Bézier surfaces, B-spline surfaces, Nonuniform Rational B-spline surfaces, Hierarchical B-Spline surfaces and T-spline surfaces are evaluated as options for the surface description method. T-spline surfaces are chosen since they are found to be superior considering the requirements of the work. In the modelling process, a modular approach is followed. Firstly, high detailed facial regions (i.e. nose, eyes,...

Citation Formats

A. Akman, Y. Sahillioğlu, and T. M. Sezgin, “Deep generation of 3D articulated models and animations from 2D stick figures,” COMPUTERS & GRAPHICS-UK, vol. 109, pp. 65–74, 2022, Accessed: 00, 2023. [Online]. Available: https://hdl.handle.net/11511/101611.