Animating physiological characteristics on 2D or 3D avatars

1. A method for applying a physiological characteristic to a portion of video, the method comprising:

receiving a frame of a video sequence;

receiving a physiological signal;

generating an attention mask based on the received physiological signal, wherein the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence;

generating a pixel adjustment value based on the physiological signal and the attention mask; and

applying the pixel adjustment value to an identified pixel in the frame of the video sequence.

2. The method of claim 1 , further comprising:

generating a second pixel adjustment value based on the physiological signal and the attention mask, the second pixel adjustment value being different from the pixel adjustment value; and

applying the second pixel adjustment value to a second identified pixel in the frame of the video sequence, wherein the second identified pixel is different from the identified pixel.

3. The method of claim 1 , wherein the attention mask identifies areas of an avatar depicted in the video sequence that are affected by the physiological signal.

4. The method of claim 1 , wherein the attention mask is generated from a machine learning model specifically trained to generate attention masks based on a frame of a video sequence and the physiological signal.

5. The method of claim 1 , wherein the physiological signal is at least one of a blood volume pulse rate, a blinking rate, or a respiratory rate.

6. The method of claim 1 , further comprising generating another pixel adjustment value based on the physiological signal, the attention mask, and color coefficients, wherein the color coefficients are specific to the physiological characteristic.

7. The method of claim 6 , wherein applying the color coefficients includes generating an alpha mask for each color of the color coefficients and combining the alpha masks to obtain an output frame.

8. The method of claim 1 , wherein the pixel adjustment value is a vector corresponding to a direction and magnitude for one or more pixels.

9. A computer storage media including instructions, which when executed by a processor, cause the processor to:

receive a frame of a video sequence;

receive a physiological signal;

generate an attention mask based on the received physiological signal, wherein the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence;

generate an alpha mask for a first color based on the frame of the video sequence, the physiological signal, the attention mask, and a color channel coefficient associated with the first color; and

combine the generated alpha mask for the first color with an alpha mask of a second color to generate an output frame.

10. The computer storage media of claim 9 , wherein the attention mask identifies areas of an avatar depicted in the video sequence that are affected by the physiological signal.

11. The computer storage media of claim 9 , wherein the physiological signal is at least one of a blood volume pulse rate, a blinking rate, or a respiratory rate.

12. The computer storage media of claim 9 , wherein the attention mask is generated from a machine learning model specifically trained to generate attention masks based on a frame of a video sequence and the physiological signal.

13. The computer storage media of claim 9 , wherein the instructions, when executed by the processor, cause the processor to receive an external factor affecting at least one of the color coefficients, attention mask, or physiological signal.

14. A system for applying a physiological characteristic to a portion of video, the system comprising:

a processor; and

memory storing instructions, which when executed by the processor, cause the processor to:

receive a frame of a video sequence;

receive a physiological signal;

generate an attention mask based on the received physiological signal, wherein the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence;

generate a pixel adjustment value based on the physiological signal and the attention mask; and

apply the pixel adjustment value to an identified pixel in the frame of the video sequence.

15. The system of claim 14 , wherein the instructions cause the processor to:

generate a second pixel adjustment value based on the physiological signal and the attention mask, the second pixel adjustment value being different from the pixel adjustment value; and

apply the second pixel adjustment value to a second identified pixel in the frame of the video sequence, wherein the second identified pixel is different from the identified pixel.

16. The system of claim 14 , wherein the attention mask identifies areas of an avatar depicted in the video sequence that are affected by the physiological signal.

17. The system of claim 16 , wherein the attention mask is generated from a machine learning model specifically trained to generate attention masks based on a frame of a video sequence and the physiological signal.

18. The system of claim 14 , wherein the instructions cause the processor to generate another pixel adjustment value based on the physiological signal, the attention mask, and color coefficients, wherein the color coefficients are specific to the physiological characteristic.

19. The system of claim 18 , wherein applying the color coefficients includes generating an alpha mask for each color of the color coefficients and combining the alpha masks to obtain an output frame.

20. The system of claim 18 , wherein the instructions cause the processor to receive an external factor affecting at least one of the color coefficients, attention mask, or physiological signal, and generate the another pixel adjustment value based on the external factor.