Movement tracking for video communications in a virtual environment
Methods, systems, and apparatus, including computer programs encoded on computer storage media relate to a method for head tracking for video communications in a virtual environment. The system may provide a video conference session in a virtual environment. The system may provide a digital representation of the video conference participant in the virtual environment. The system may display one or more views of the virtual environment in the video conference. The system may track movement of the video conference participant to generate user movement information and may display movement of the video conference participant on the digital representation.
1 . A method comprising:
receiving a video stream of a video conference participant in a virtual environment;
detecting, within the video stream, movement of the video conference participant performing an act of viewing past a boundary;
generating movement information indicating the act of viewing past the boundary; and
triggering an action for updating a digital representation of the video conference participant based on the movement information.
2 . The method of claim 1 , wherein the virtual environment comprises one of: (i) a virtual reality (VR) environment including 3D representations of one or more users and (ii) an augmented reality (AR) environment comprising one or more AR holograms.
3 . The method of claim 1 , further comprising: providing a captured left view of the virtual environment and a captured right view of the virtual environment as part of the video stream or in a second video stream in a video conference session.
4 . The method of claim 1 , further comprising:
turning a head of the digital representation of the video conference participant by an amount corresponding to an amount of a turn of the head of the video conference participant.
5 . The method of claim 1 , further comprising:
turning a body of the digital representation of the video conference participant based on the movement information.
6 . The method of claim 1 , wherein the digital representation of the video conference participant comprises a flat shape in the virtual environment displaying at least a portion of the video stream of the video conference participant.
7 . The method of claim 1 , wherein detecting movement that corresponds with the video conference participant performing an act of viewing past a boundary comprises:
identifying a direction of movement of the act; and
wherein generating movement information that indicates the act of viewing past the boundary comprises:
associating a direction attribute with the movement information based on the identified direction of movement of the act.
8 . The method of claim 1 , wherein triggering an action for updating the digital representation comprises:
updating a current viewpoint of the digital representation based at least on a direction of the movement information, wherein the boundary comprises: a current physical placement of an edge relative to a screen displaying at least a portion of a respective view of the virtual environment.
9 . The method of claim 1 , wherein a left view of the virtual environment left of the digital representation is a different view than a right view of the virtual environment right of the digital representation.
10 . An apparatus, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
receive a video stream of a video conference participant in a virtual environment;
detect, within the video stream, movement of the video conference participant performing an act of viewing past a boundary;
generate movement information indicating the act of viewing past the boundary; and
trigger an action for updating a digital representation of the video conference participant based on the movement information.
11 . The apparatus of claim 10 , wherein the processor is further configured to execute instructions stored in the memory to:
provide a captured left view of the virtual environment and a captured right view of the virtual environment as part of the video stream or in a second video stream in a video conference session.
12 . The apparatus of claim 10 , wherein the processor is further configured to execute instructions stored in the memory to:
turn a head of the digital representation of the video conference participant by an amount corresponding to an amount of a turn of the head of the video conference participant.
13 . The apparatus of claim 10 , wherein the processor is further configured to execute instructions stored in the memory to:
turn a body of the digital representation of the video conference participant based on the movement information.
14 . The apparatus of claim 10 , wherein the digital representation of the video conference participant comprises a flat shape in the virtual environment displaying at least a portion of the video stream of the video conference participant.
15 . A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:
receiving a video stream of a video conference participant in a virtual environment;
detecting, within the video stream, movement of the video conference participant performing an act of viewing past a boundary;
generating movement information indicating of the act of viewing past the boundary; and
triggering an action for updating a digital representation of the video conference participant based on the movement information.
16 . The non-transitory computer readable medium of claim 15 , wherein the virtual environment comprises a VR environment including 3D representations of one or more users.
17 . The non-transitory computer readable medium of claim 15 , wherein the processors are further configured to perform the operations of:
turning a head of the digital representation of the video conference participant by an amount corresponding to a turn of the head of the video conference participant.
18 . The non-transitory computer readable medium of claim 15 , wherein the processors are further configured to perform the operations of:
turning a body of the digital representation of the video conference participant based on detected movement information.
19 . The non-transitory computer readable medium of claim 15 , wherein the digital representation of the video conference participant comprises a flat shape in the virtual environment displaying at least a portion of the video stream of the video conference participant.
20 . The non-transitory computer readable medium of claim 15 , wherein detecting movement that corresponds with the video conference participant performing an act of viewing past a boundary comprises:
identifying a direction of movement of the act; and
wherein generating movement information indicating the act of viewing past the boundary comprises:
associating a direction attribute with the movement information based on the identified direction of movement of the act.