Gaze-tracking-based image downscaling for multi-party video communication

1. A method for facilitating video communication by predicting shared locations during a concurrent video presentation, comprising:

receiving a first sequence of gaze locations from a first receiving computing device and a second sequence of gaze locations from a second receiving computing device during concurrent video communication sessions between the first and second receiving computing devices and a presenting computing device;

predicting a shared location based on the first sequence of gaze locations and the second sequence of gaze locations;

generating a mask comprising a first region based on the shared location, wherein the first region does not include the shared location;

generating, based on the mask, a downscaled encoding of a source image from the presenting computing device by downscaling the source image at the first region of the source image; and

sending the downscaled encoding to the first receiving computing device and the second receiving computing device.

2. The method of claim 1 , wherein predicting the shared location comprising:

determining a first future gaze location based on the first sequence of gaze locations;

determining a second future gaze location based on the second sequence of gaze locations;

determining a centroid based on the first and second future gaze locations; and

determining the shared location based on the centroid.

3. The method of claim 2 , wherein predicting the shared location comprising:

obtaining a user record associated with the first sequence of gaze locations, wherein the user record stores a plurality of gaze scanpaths; and

training a neural network based on the plurality of gaze scanpaths to predict future scanpath predictions, wherein determining the first future gaze location comprises determining the first future gaze location using the neural network.

4. The method of claim 1 , further comprising determining whether a data rate of the downscaled encoding satisfies a data rate threshold, wherein sending the downscaled encoding comprises sending the downscaled encoding in response to a determination that the data rate satisfies the data rate threshold.

5. The method of claim 4 , further comprising determining the data rate of the downscaled encoding by determining an upload rate of the presenting computing device.

6. One or more tangible, non-transitory, machine-readable media storing instructions that, when executed by one or more processors, effectuate operations comprising:

receiving a first set of screen positions from a first receiving computing device and a second set of screen positions from a second receiving computing device during concurrent video communication sessions between the first and second receiving computing devices and a presenting computing device;

predicting a shared position based on the first set of screen positions and the second set of screen positions;

generating a downscaled encoding of a source image from the presenting computing device by downscaling the source image at a set of regions of the source image, wherein the set of regions does not comprise the shared position; and

sending the downscaled encoding to the first receiving computing device and the second receiving computing device.

7. The media of claim 6 , wherein:

the first receiving computing device determines a first set of head orientations and a first set of eye measurements to determine the first set of screen positions; and

the second receiving computing device determines a second set of head orientations and a second set of eye measurements to determine the second set of screen positions.

8. The media of claim 6 , wherein:

receiving the first set of screen positions comprises receiving categorical indicators associated with the first set of screen positions; and

each respective indicator of the categorical indicators indicates whether a respective screen position of the first set of screen positions is associated with a saccade position or a gaze position; and

predicting the shared position comprises determining the shared position based on the categorical indicators.

9. The media of claim 6 , wherein predicting the shared position comprises:

determining a set of density-based clustering values based on a cluster of screen positions comprising the first set of screen positions and the second set of screen positions; and

determining the shared position based on the set of density-based clustering values.

10. The media of claim 6 , wherein:

each respective screen position of the first set of screen positions and the second set of screen positions is associated with a respective measurement time of a set of measurement times; and

predicting the shared position comprises providing a machine learning model with the set of measurement times to determine the shared position.

11. The media of claim 6 , the operations further comprising receiving a third screen position from a third receiving computing device during the concurrent video communication sessions, wherein:

the concurrent video communication sessions comprise a communication session between the third receiving computing device and the presenting computing device; and

predicting the shared position comprises predicting the shared position based on the third screen position.

12. The media of claim 11 , wherein the downscaled encoding is a first downscaled encoding, and wherein the source image is a first source image, and wherein the set of regions is a first set of regions, the operations further comprising:

receiving a fourth screen position from the first receiving computing device;

receiving a fifth screen position from the second receiving computing device;

receiving a sixth screen position from the third receiving computing device;

determining that the sixth screen position exceeds a screen boundary;

predicting a second shared position based on an additional set of screen positions, wherein the additional set of screen positions comprises the fourth screen position and the fifth screen position, and wherein the additional set of screen positions does not comprise the sixth screen position; and

generating a second downscaled encoding of a second source image from the presenting computing device by downscaling the second source image at a second set of regions of the source image, wherein the second set of regions does not comprise the second shared position; and

sending the second downscaled encoding to the first receiving computing device and the second receiving computing device.

13. The media of claim 6 , wherein downscaling the source image at a set of regions of the source image comprises interpolating a set of pixel color values of the source image.

14. A system comprising:

one or more processors; and

memory storing computer program instructions that, when executed by the one or more processors, cause the one or more processors to effectuate operations comprising:

obtaining a first set of screen positions associated with a first receiving computing device and a second set of screen positions associated with a second receiving computing device during concurrent video communication sessions between a presenting computing device and the first and second receiving computing devices;

predicting a shared position based on the first set of screen positions and the second set of screen positions;

downscaling a source image from the presenting computing device at a set of regions of the source image to generate a downscaled encoding, wherein the set of regions does not comprise the shared position; and

sending the downscaled encoding to the first receiving computing device and the second receiving computing device.

15. The system of claim 14 , wherein the set of regions is a first set of regions, wherein predicting the shared position comprises:

detecting a set of features in the source image; and

generating a second set of regions corresponding to the set of features, wherein the first set of regions does not comprise the second set of regions.

16. The system of claim 15 , wherein detecting the set of features comprises:

receiving a first indication that the source image is being displayed;

in response to receiving the first indication, providing the source image to as an input to a neural network; and

indicating the set of features based on an output of the neural network.

17. The system of claim 15 , wherein detecting the set of features in the source image comprises:

determining a presentation category associated with the source image;

selecting a first set neural network model parameters based on the presentation category; and

detecting the set of features in the source image using the first set neural network model parameters.

18. The system of claim 15 , wherein the set of features comprises text of the source image.

19. The system of claim 14 , wherein:

obtaining the first set of screen positions comprises obtaining a first stored set of screen positions of a first user associated with the first receiving computing device;

obtaining the second set of screen positions comprises obtaining a second stored set of screen positions of a second user associated with the second receiving computing device;

training a first neural network model to predict a first set of future screen positions based on the first stored set of screen positions; and

training a second neural network model to predict a second set of future screen positions based on the second stored set of screen positions, wherein predicting the shared position comprises:

predicting a first predicted position using the first neural network model after the training of the first neural network model;

predicting a second predicted position using the second neural network model after the training of the second neural network model; and

determining the predicted shared position based on the first and second predicted positions.

20. The system of claim 14 , wherein generating the downscaled encoding comprises:

determining a set of values representing pixel data of the source image;

providing the set of values to a first convolutional neural network layer as an input of the first convolutional neural network;

using a first layer output of the first convolutional neural network layer as an input for a second convolutional neural network layer;

using both a second layer output of the second convolutional neural network layer and the first layer output as inputs for a third convolutional neural network layer; and

generating the downscaled encoding based on an output of the third convolutional neural network layer.