Simplified annotation-free domain transfer for facial avatars

1 . An apparatus for generating an image, comprising:

at least one memory; and

at least one processor coupled to the at least one memory, the at least one processor being configured to:

obtain a first image, wherein the first image is associated with a first image domain;

convert, using a machine learning (ML) model, the first image into a second image domain to generate a second image;

generate a first set of domain agnostic features for the first image;

generate a second set of domain agnostic features for the second image;

determine a loss value based on a comparison of the first set of domain agnostic features and the second set of domain agnostic features;

adjust the ML model based on the loss value; and

generate a third image in the second image domain based on the adjusted ML model.

2 . The apparatus of claim 1 , wherein the ML model is trained further based on:

determine an adversarial loss value based on a comparison between the second image and a fourth image, wherein the fourth image comprises a real image captured using a sensor in the second image domain; and

adjust the ML model further based on the adversarial loss value.

3 . The apparatus of claim 1 , wherein the at least one processor is configured to determine a region of interest (ROI) based on the first image, and wherein the loss value is determined for the ROI.

4 . The apparatus of claim 3 , wherein the ROI is determined based on a foreground of the first image.

5 . The apparatus of claim 4 , wherein the first image is rendered based on a 3D model, and wherein the foreground is based on portions of the 3D model in the first image.

6 . The apparatus of claim 3 , wherein training the ML model causes the ML model to determine weights for pixels of the ROI, and wherein the loss value is determined based on the weights.

7 . The apparatus of claim 6 , wherein the weights are determined based on a degree of freedom associated with a portion of the image corresponding to the pixels of the ROI.

8 . The apparatus of claim 1 , wherein the first image domain comprises a color image domain, and wherein the second image domain comprises an infrared image domain.

9 . The apparatus of claim 8 , wherein the infrared image domain includes a near-infrared (NIR) image domain.

10 . The apparatus of claim 1 , wherein the apparatus further comprises a camera for capturing a fifth image in the first image domain, wherein the third image is generated based on the fifth image using the adjusted ML model.

11 . A method for generating an image, comprising:

obtaining a first image, wherein the first image is associated with a first image domain;

converting, using a machine learning (ML) model, the first image into a second image domain to generate a second image;

generating a first set of domain agnostic features for the first image;

generating a second set of domain agnostic features for the second image;

determining a loss value based on a comparison of the first set of domain agnostic features and the second set of domain agnostic features;

adjusting the ML model based on the loss value; and

generating a third image in the second image domain based on the adjusted ML model.

12 . The method of claim 11 , further comprising:

determining an adversarial loss value based on a comparison between the second image and a fourth image, wherein the fourth image comprises a real image captured using a sensor in the second image domain; and

adjusting the ML model further based on the adversarial loss value.

13 . The method of claim 11 , further comprising determining a region of interest (ROI) based on the first image, and wherein the loss value is determined for the ROI.

14 . The method of claim 13 , wherein the ROI is determined based on a foreground of the first image.

15 . The method of claim 14 , wherein the first image is rendered based on a 3D model, and wherein the foreground is based on portions of the 3D model in the first image.

16 . The method of claim 13 , further comprising determining weights for pixels of the ROI, and wherein the loss value is determined based on the weights.

17 . The method of claim 16 , wherein the weights are determined based on a degree of freedom associated with a portion of the image corresponding to the pixels of the ROI.

18 . The method of claim 11 , wherein the first image domain comprises a color image domain, and wherein the second image domain comprises an infrared image domain.

19 . The method of claim 18 , wherein the infrared image domain includes a near-infrared (NIR) image domain.

20 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:

obtain a first image, wherein the first image is associated with a first image domain;

convert, using a machine learning (ML) model, the first image into a second image domain to generate a second image;

generate a first set of domain agnostic features for the first image;

generate a second set of domain agnostic features for the second image;

determine a loss value based on a comparison of the first set of domain agnostic features and the second set of domain agnostic features;

adjust the ML model based on the loss value; and

generate a third image in the second image domain based on the adjusted ML model.