Changing directions of gaze during a video conference
A system may identify, during a video conference, a first participant based on a direction of a gaze of a second participant toward a user tile associated with the first participant within a graphical user interface of the video conference. The system may cause a change, in a video stream transmitted to a device of the first participant during the video conference, to the direction of the gaze of the second participant to appear directed toward a camera of a device of the second participant. In some implementations, the system may cause a change, in a second video stream transmitted to a device of a third participant, to the direction of the gaze of the second participant to appear directed toward the user tile.
1 . A method, comprising:
identifying, during a video conference, a first participant based on a direction of a gaze of a second participant toward a user tile associated with the first participant within a graphical user interface (GUI) of the video conference; and
causing a change, in a video stream transmitted to a device of the first participant during the video conference, to the direction of the gaze of the second participant to appear directed toward a camera of a device of the second participant, wherein the change comprises generating a personalized video stream for the first participant such that a gaze correction is applied in the video stream transmitted to the device of the first participant and not transmitted to other participant devices;
wherein the personalized video stream is generated by digitally correcting, for each recipient participant, the gaze of the second participant in a composite video stream based on a user tile position of the recipient in the GUI, such that the gaze correction is specific to a view of the recipient and is not present in video streams transmitted to other participants.
2 . The method of claim 1 , further comprising:
causing a change, in a second video stream transmitted to a device of a third participant, to the direction of the gaze of the second participant to appear directed toward the user tile.
3 . The method of claim 1 , further comprising:
calculating, from pupils of the second participant, an intersection point at the GUI; and
correlating the intersection point to the user tile.
4 . The method of claim 1 , further comprising:
using a machine learning model trained on historical gaze intersection data from prior video conferences to determine an intersection point at the GUI based on the gaze; and
correlating the intersection point to a position of the user tile.
5 . The method of claim 1 , wherein causing the change comprises:
selecting the camera, from a plurality of cameras of the device of the second participant, during the video conference, based on a calculated intersection point, such that the selected camera is physically closest to the user tile as displayed in the GUI.
6 . The method of claim 1 , wherein causing the change comprises:
correcting, digitally in a user tile associated with the second participant, pupils of the second participant.
7 . The method of claim 1 , further comprising:
causing a change, in a second video stream transmitted to a device of a third participant, to a 3D representation of the second participant to appear directed toward a 3D representation of the first participant.
8 . The method of claim 1 , further comprising:
outputting a plurality of user tiles, in the GUI, in a gallery layout; and
maintaining a data structure indicating X and Y positions of user tiles in the gallery layout during the video conference.
9 . A system, comprising:
a memory subsystem; and
processing circuitry configured to execute instructions stored in the memory subsystem to:
identify, during a video conference, a first participant based on a direction of a gaze of a second participant toward a user tile associated with the first participant within a graphical user interface (GUI) of the video conference; and
cause a change, in a video stream transmitted to a device of the first participant during the video conference, to the direction of the gaze of the second participant to appear directed toward a camera of a device of the second participant, wherein the change comprises a generation of a personalized video stream for the first participant such that a gaze correction is applied in the video stream transmitted to the device of the first participant and not transmitted to other participant devices;
wherein the personalized video stream is generated by digitally correcting for each recipient participant, the gaze of the second participant in a composite video stream based on a user tile position of the recipient in the GUI, such that the gaze correction is specific to a view of the recipient and is not present in video streams transmitted to other participants.
10 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:
arrange a plurality of user tiles, including the user tile associated with the first participant, in a gallery layout.
11 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:
correct, digitally in a user tile associated with the second participant, a head of the second participant.
12 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:
determine the user tile based on an intersection point at the GUI from pupils of the second participant.
13 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:
train a machine learning model, using a dataset of past intersection points and associated user tile selections.
14 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:
change from a first camera that is a default camera during the video conference to the camera based on the camera being closest to the user tile.
15 . A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:
identifying, during a video conference, a first participant based on a direction of a gaze of a second participant toward a user tile associated with the first participant within a graphical user interface (GUI) of the video conference; and
causing a change, in a video stream transmitted to a device of the first participant during the video conference, to the direction of the gaze of the second participant to appear directed toward a camera of a device of the second participant, wherein the change comprises generating a personalized video stream for the participant such that a gaze correction is applied in the video stream transmitted to the device of the first participant and not transmitted to other participant devices;
wherein the personalized video stream is generated by digitally correcting, for each recipient participant, the gaze of the second participant in a composite video stream based on a user tile position of the recipient in the GUI, such that the gaze correction is specific to a view of the recipient and is not present in video streams transmitted to other participants.
16 . The non-transitory computer readable medium of claim 15 , wherein the video stream is a first composite video stream, transmitted to the device of the first participant, different from a second composite video stream transmitted to the device of the second participant.
17 . The non-transitory computer readable medium of claim 15 , wherein the operations further comprise:
selecting a default camera for the video conference; and
changing from the default camera to the camera based on the camera being closest to the user tile.
18 . The non-transitory computer readable medium of claim 15 , wherein the operations further comprise:
causing a change, in a second video stream transmitted to a device of a third participant, to a 3D representation of the second participant to appear directed toward a 3D representation of the first participant.
19 . The non-transitory computer readable medium of claim 15 , wherein the operations further comprise:
outputting a plurality of user tiles, in the GUI, in a gallery layout; and
maintaining a data structure indicating Cartesian coordinates of user tiles in the gallery layout during the video conference.
20 . The non-transitory computer readable medium of claim 15 , wherein the operations further comprise:
correcting, digitally in a user tile associated with the second participant, a body of the second participant.