3D hair synthesis using volumetric variational autoencoders

1. A method for single-view 3D hair modeling, the method comprising:

training, by a neural network processor, a volumetric autoencoder to encode a plurality of 3D hairstyles into latent features by representing the plurality of 3D hairstyles using an occupancy field representing a volume of a specific hairstyle and a flow field representing an orientation of hair strands of the specific hairstyle, and to generate an output based on the latent features;

training, by the neural network processor, an embedding network to determine hair coefficients of a single hairstyle from an input image;

receiving, by the neural network processor, the input image; and

synthesizing, by the neural network processor, hair strands to generate a single-view 3D model of the single hairstyle based on the volumetric autoencoder, the embedding network, and the input image.

2. The method of claim 1 , wherein the flow field is determined by averaging orientations of nearby strands and diffusing the flow field into the volume represented by the occupancy field.

3. The method of claim 1 , wherein training the volumetric autoencoder includes training the volumetric autoencoder to include a volumetric encoder to encode the plurality of 3D hairstyles into the latent features and an occupancy field decoder to decode the occupancy field representing the volume of the single hairstyle and an orientation field decoder to decode the flow field representing the orientation of hair strands of the single hairstyle to generate the output.

4. The method of claim 1 , further comprising performing Principled Component Analysis (PCA) on the hair coefficients.

5. The method of claim 1 , further comprising performing, by the neural network processor, post-processing on the single-view 3D model of the single hairstyle to place the single-view 3D model of the single hairstyle on at least one of a face or a head.

6. The method of claim 5 , wherein the post-processing includes:

segmenting a pixel-level hair mask and a digitized head model starting from the input image; and

applying a mask-based deformation method to improve alignment with the hair mask.

7. A device for single-view 3D hair modeling, the device comprising:

a neural network processor; and

a memory, coupled to the neural network processor, the memory including instructions causing the neural network processor to:

train a volumetric autoencoder to encode a plurality of 3D hairstyles into latent features by representing the plurality of 3D hairstyles using an occupancy field representing a volume of a specific hairstyle and a flow field representing an orientation of hair strands of the specific hairstyle, and to generate an output based on the latent features;

train an embedding network to determine hair coefficients of a single hairstyle from an input image;

receive the input image; and

synthesize hair strands to generate a single-view 3D model of the single hairstyle based on the volumetric autoencoder, the embedding network, and the input image.

8. The device of claim 7 , wherein the flow field is determined by averaging orientations of nearby strands and diffusing the flow field into the volume represented by the occupancy field.

9. The device of claim 7 , wherein training the volumetric autoencoder includes training the volumetric autoencoder to include a volumetric encoder to encode the plurality of 3D hairstyles into the latent features and an occupancy field decoder to decode the occupancy field representing the volume of the single hairstyle and an orientation field decoder to decode the flow field representing the orientation of hair strands of the single hairstyle to generate the output.

10. The device of claim 7 , the memory including instructions further causing the neural network processor to perform Principled Component Analysis (PCA) on the hair coefficients.

11. The device of claim 7 , the memory including instructions further causing the neural network processor to perform post-process on the single-view 3D model of the single hairstyle to place the single-view 3D model of the single hairstyle on at least one of a face or a head.

12. The device of claim 11 , wherein the post-processing includes:

segmenting a pixel-level hair mask and a digitized head model starting from the input image; and

applying a mask-based deformation method to improve alignment with the hair mask.

13. A method for single-view 3D hair modeling, the method comprising:

encoding, using a volumetric autoencoder, a plurality of 3D hairstyles into latent features by representing the plurality of 3D hairstyles using an occupancy field representing a volume of a specific hairstyle and a flow field representing an orientation of hair strands of the specific hairstyle;

generating an output based on the latent features;

determining hair coefficients of a single hairstyle from an input image;

receiving the input image; and

generating a single view 3D model of the single hairstyle based on the volumetric autoencoder, an embedding network and the input image.

14. The method of claim 13 , wherein the flow field is determined by averaging orientations of nearby strands and diffusing the flow field into the volume represented by the occupancy field.

15. The method of claim 13 , wherein encoding a plurality of 3D hairstyles into latent features includes training the volumetric autoencoder to include a volumetric encoder to encode the plurality of 3D hairstyles into the latent features and an occupancy field decoder to decode the occupancy field representing the volume of the single hairstyle and an orientation field decoder to decode the flow field representing the orientation of hair strands of the single hairstyle to generate the output.

16. The method of claim 13 , further comprising performing Principled Component Analysis (PCA) on the hair coefficients.

17. The method of claim 13 , further comprising placing the single-view 3D model of the single hairstyle on at least one of a face or a head.