3D dataset generation for neural network model training

1. An electronic device, comprising:

circuitry configured to:

receive a set of two-dimensional (2D) images comprising at least a first two-dimensional (2D) image of an object of interest;

detect a plurality of 2D landmarks on the first 2D image, wherein the detected plurality of 2D landmarks corresponds to shape-features of the object of interest;

align a first three-dimensional (3D) shape model of a reference object by fitting corresponding 2D landmarks of the first 3D shape model to the detected plurality of 2D landmarks of the object of interest;

estimate texture mapping information between the object of interest of the first 2D image and the aligned first 3D shape model;

generate a dataset by including the first 2D image, the aligned first 3D shape model, and the estimated texture mapping information as a first sample of the dataset; and

train, based on the generated dataset, a neural network model on a task of 3D reconstruction from a single 2D image, wherein the trained neural network model:

receives, as an input, a second 2D image of the object of interest, wherein the second 2D image is part of an unseen dataset of images of the object of interest; and

outputs, based on the input, a second 3D shape model corresponding to the object of interest in the second 2D image.

2. The electronic device according to claim 1 , wherein the circuitry is further configured to:

extract a plurality of 3D landmarks from the aligned first 3D shape model; and

generate the dataset by including the detected plurality of 2D landmarks and the extracted plurality of 3D landmarks in the first sample of the dataset.

3. The electronic device according to claim 2 , wherein each 3D landmark of the extracted plurality of 3D landmarks comprises 3D coordinates of a 3D point on a corresponding shape-feature of the aligned first 3D shape model.

4. The electronic device according to claim 1 , wherein the first 3D shape model is one of a 3D morphable model or a 3D Basel model.

5. The electronic device according to claim 1 , wherein the object of interest is one of an inanimate object or an animate object.

6. The electronic device according to claim 1 , wherein the object of interest is one of a body of a human subject, a face of the human subject, or an anatomical portion different from the body and the face of the human subject.

7. The electronic device according to claim 1 , wherein the circuitry is further configured to solve an objective function to minimize a distance between the corresponding 2D landmarks of the first 3D shape model and the detected plurality of 2D landmarks of the object of interest, wherein the fitting corresponds to the solving of the objective function.

8. A method, comprising:

in an electronic device:

receiving a set of two-dimensional (2D) images comprising at least a first two-dimensional (2D) image of an object of interest;

detecting a plurality of 2D landmarks on the first 2D image, wherein the detected plurality of 2D landmarks corresponds to shape-features of the object of interest;

aligning a first three-dimensional (3D) shape model of a reference object by fitting corresponding 2D landmarks of the first 3D shape model to the detected plurality of 2D landmarks of the object of interest;

estimating texture mapping information between the object of interest of the first 2D image and the aligned first 3D shape model;

generating a dataset by including the first 2D image, the aligned first 3D shape model, and the estimated texture mapping information as a first sample of the dataset;

training, based on the generated dataset, a neural network model on a task of 3D reconstruction from a single 2D image;

receiving, by the trained neural network model, a second 2D image of the object of interest as an input, wherein the second 2D image is part of an unseen dataset of images of the object of interest; and

outputting, by the trained neural network model, a second 3D shape model corresponding to the object of interest in the second 2D image, based on the input.

9. The method according to claim 8 , further comprising:

extracting a plurality of 3D landmarks from the aligned first 3D shape model; and

generating the dataset by including the detected plurality of 2D landmarks and the extracted plurality of 3D landmarks in the first sample of the dataset.

10. The method according to claim 9 , wherein each 3D landmark of the extracted plurality of 3D landmarks comprises 3D coordinates of a 3D point on a corresponding shape-feature of the aligned first 3D shape model.

11. The method according to claim 8 , wherein the first 3D shape model is one of a 3D morphable model or a 3D Basel model.

12. The method according to claim 8 , wherein the object of interest is one of an inanimate object or an animate object.

13. The method according to claim 8 , wherein the object of interest is one of a body of a human subject, a face of the human subject, or an anatomical portion different from the body and the face of the human subject.

14. The method according to claim 8 , further comprising solving an objective function to minimize a distance between the corresponding 2D landmarks of the first 3D shape model and the detected plurality of 2D landmarks of the object of interest, wherein the fitting corresponds to the solving of the objective function.

15. A non-transitory computer-readable medium having stored thereon computer-implemented instructions that, when executed by an electronic device, causes the electronic device to execute operations, the operations comprising:

receiving a set of two-dimensional (2D) images comprising at least a first two-dimensional (2D) image of an object of interest;

detecting a plurality of 2D landmarks on the first 2D image, wherein the detected plurality of 2D landmarks corresponds to shape-features of the object of interest;

aligning a first three-dimensional (3D) shape model of a reference object by fitting corresponding 2D landmarks of the first 3D shape model to the detected plurality of 2D landmarks of the object of interest;

estimating texture mapping information between the object of interest of the first 2D image and the aligned first 3D shape model;

generating a dataset by including the first 2D image, the aligned first 3D shape model, and the estimated texture mapping information as a first sample of the dataset;

training, based on the generated dataset, a neural network model on a task of 3D reconstruction from a single 2D image;

receiving, by the trained neural network model, a second 2D image of the object of interest as an input, wherein the second 2D image is part of an unseen dataset of images of the object of interest; and

outputting, by the trained neural network model, a second 3D shape model corresponding to the object of interest in the second 2D image, based on the input.

16. The non-transitory computer-readable medium according to claim 15 , wherein the operations further comprises:

extracting a plurality of 3D landmarks from the aligned first 3D shape model; and

generating the dataset by including the detected plurality of 2D landmarks and the extracted plurality of 3D landmarks in the first sample of the dataset.

17. The non-transitory computer-readable medium according to claim 15 , wherein the first 3D shape model is one of a 3D morphable model or a 3D Basel model.

18. The non-transitory computer-readable medium according to claim 15 , wherein the object of interest is one of an inanimate object or an animate object.