Generating physio-realistic avatars for training non-contact models to recover physiological characteristics

1. A method for generating video sequences including physio-realistic avatars, the method comprising:

receiving an albedo for an avatar, wherein the albedo represents a texture map including skin pixels associated with the avatar;

modifying a sub-surface skin color associated with the skin pixels of the albedo based on a sub-surface skin color weight applied to physiological data associated with physiologic characteristic, wherein the sub-surface skin color weight is based on an absorption spectrum associated with the physiological data for one or more color channels;

rendering the avatar based on the albedo and the modified sub-surface skin color; and

synthesizing a frame of video, the frame of video including the avatar.

2. The method of claim 1 , wherein the physiological data varies with time, the method further comprising:

modifying the sub-surface skin color associated with the albedo based on a combination of a base sub-surface skin color and the sub-surface skin color weight applied to the physiological data at a first time;

rendering the avatar based on the albedo and the modified sub-surface skin color associated with the physiological data at the first time;

synthesizing a first frame of video, the first frame of video including the avatar rendered based on the albedo and the modified sub-surface skin color associated with the physiological data at the first time;

modifying the sub-surface skin color associated with the albedo based on the physiological data at a second time;

rendering the avatar based on the albedo and the modified sub-surface skin color associated with the physiological data at the second time; and

synthesizing a second frame of video, the second frame of video including the avatar rendered based on the albedo and the modified sub-surface skin color associated with the physiological data at the second time.

3. The method of claim 1 , further comprising:

modifying the physiological data with weighting factors specific to the physiological data; and

modifying the sub-surface skin color associated with the albedo with the modified physiological data.

4. The method of claim 3 , further comprising varying a sub-surface radii for the one or more color channels based on the weighting factors specific to the physiological data.

5. The method of claim 1 , further comprising training a machine learning model with a plurality of synthesized frames of that include the avatar.

6. The method of claim 5 , further comprising training the machine learning model with a plurality of videos including human subjects.

7. The method of claim 6 , further comprising:

receiving a plurality of video frames depicting a human subject; and

recovering a physiological signal based on the trained machine learning model.

8. The method of claim 1 , wherein the frame of video includes the avatar in front of a dynamic background.

9. The method of claim 1 , further comprising:

receiving the physiological data from a requesting entity;

synthesizing the frame of video including the avatar substantially in real-time; and

providing the frame of video to the requesting entity.

10. The method of claim 1 , wherein the physiologic characteristic is blood volume pulse.

11. The method of claim 1 , further comprising labeling a video segment including the synthesized frame of video with a training label specific to the physiologic characteristic.

12. A system for training a machine learning model using video sequences including physio-realistic avatars, the system comprising:

a processor; and

memory storing instructions, which when executed by the processor, cause the processor to:

receive a request from a requesting entity to train the machine learning model to detect a physiological characteristic;

receive a plurality of video segments, wherein one or more of the plurality of video segments includes a synthetic physio-realistic avatar generated with the physiological characteristic, and wherein the synthetic physio-realistic avatar includes color information associated with the physiological characteristic corresponding to at least a part of skin pixels of the synthetic physio-realistic avatar, wherein the color information includes a sub-surface skin color associated with skin pixels of an albedo, wherein the sub-surface skin color is modified according to a sub-surface skin color weight, and wherein the sub-surface skin color weight is based on an absorption spectrum based on physiological data of the physiological characteristic for one or more color channels;

train the machine learning model with the plurality of video segments; and

provide a trained model to the requesting entity.

13. The system of claim 12 , further comprising instructions, which when executed by the processor, cause the processor to:

receive a second plurality of video segments, wherein one or more video segments included in the second plurality of video segments depicts a human with the physiological characteristic; and

train the machine learning model with the plurality of video segments and the second plurality of video segments.

14. The system of claim 12 , wherein the physiological characteristic is a blood volume pulse.

15. The system of claim 12 , wherein one or more of the plurality of video segments are labeled with a training label based on the physiological characteristic.

16. The system of claim 12 , further comprising instructions, which when executed by the processor, cause the processor to:

receive a second video segment;

identify the physiological characteristic from the second video segment using the trained model; and

provide an assessment of the physiological characteristic to the requesting entity.

17. A non-transitory computer-readable media including instructions, which when executed by a processor, cause the processor to:

receive a request to recover a physiological characteristic from a video segment;

obtain a machine learning model trained with training data that includes synthetic physio-realistic avatars generated with the physiological characteristic and color information associated with the physiological characteristic corresponding to at least a part of skin pixels of the synthetic physio-realistic avatars, wherein the color information includes a sub-surface skin color associated with skin pixels of an albedo for an avatar of the synthetic physio-realistic avatars, the sub-surface skin color is modified according to a sub-surface skin color weight, and the sub-surface skin color weight is based on an absorption spectrum based on physiological data of the physiological characteristic for one or more color channels;

receive the video segment;

identify a measure associated with the physiological characteristic from the video segment using the trained machine learning model; and

provide an assessment of the physiological characteristic to a requesting entity based on the measure.

18. The non-transitory computer-readable media of claim 17 , wherein the instructions, which when executed by the processor, cause the processor to:

receive the albedo for the avatar, wherein the albedo represents a texture map including the skin pixels of the albedo for the avatar;

render the avatar based on the albedo and the modified sub-surface skin color;

synthesize a frame of video, the frame of video including the avatar; and

train the machine learning model with the synthesized frame of video.

19. The non-transitory computer-readable media of claim 17 , wherein the assessment of the physiological characteristic is a pulse rate.

20. The non-transitory computer-readable media of claim 17 , wherein the received video segment depicts a human subject.