Three-dimensional modeling inside a virtual video conferencing environment with a navigable avatar, and applications thereof
Disclosed herein is a web-based videoconference system that allows for video avatars to navigate within the virtual environment. The system has a presented mode that allows for a presentation stream to be texture mapped to a presenter screen situated within the virtual environment. The relative left-right sound is adjusted to provide sense of an avatar's position in a virtual space. The sound is further adjusted based on the area where the avatar is located and where the virtual camera is located. Video stream quality is adjusted based on relative position in a virtual space. Three-dimensional modeling is available inside the virtual video conferencing environment.
1. A computer-implemented method for streaming video for a virtual video conference, comprising:
receiving a three-dimensional model of a virtual environment;
receiving a first mesh representing a first three-dimensional model of an object;
receiving a video stream of a first participant of the virtual video conference, the video stream including a plurality of frames;
generating an avatar navigable by the first participant, wherein the avatar corresponds to a second three-dimensional model represented by a second mesh;
mapping respective frames of the plurality of frames from the video stream onto the second three-dimensional model to place the video stream on the avatar, wherein the second mesh is generated independently of the video stream;
from a perspective of a virtual camera of a second participant, rendering for display for the second participant the mapped avatar and the first mesh representing the first three-dimensional model of the object within the virtual environment; and
demonstrating the object by:
simultaneously rendering the first three-dimensional model of the object in the virtual environment and presenting a second video stream of the first three-dimensional model of the object.
2. The method of claim 1 , wherein the object is a product, and wherein the second participant can navigate the virtual camera around the first three-dimensional model of the product.
3. The method of claim 1 , wherein the object is an advertisement.
4. The method of claim 1 , wherein the virtual environment is a building.
5. The method of claim 1 , further comprising:
transmitting a request, for the first mesh, from the second participant, wherein the receiving the first mesh occurs in response to the request.
6. The method of claim 1 , further comprising:
receiving a position and direction of the first participant in the three-dimensional virtual space;
receiving the video stream captured from a camera on a device of the first participant, the camera positioned to capture photographic images of the first participant;
wherein the rendering comprises rendering the virtual conference to include a mapped three-dimensional model of the avatar positioned at the position and oriented at the direction.
7. A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for streaming video for a virtual conference, comprising:
receiving a three-dimensional model of a virtual environment;
receiving a first mesh representing a first three-dimensional model of an object;
receiving a video stream of a first participant of the virtual video conference, the video stream including a plurality of frames;
generating an avatar navigable by the first participant, wherein the avatar corresponds to a second three-dimensional model represented by a second mesh;
mapping respective frames of the plurality of frames from the video stream onto the second three-dimensional model to place the video stream on the avatar, wherein the second mesh is generated independently of the video stream; and
from a perspective of a virtual camera of a second participant, rendering for display for the second participant the mapped avatar and the first mesh representing the first three-dimensional model of the object within the virtual environment
demonstrating the object by:
simultaneously rendering the first three-dimensional model of the object in the virtual environment and presenting a second video stream of the first three-dimensional model of the object.
8. The non-transitory, tangible computer-readable device of claim 7 , wherein the object is a product, and wherein the second participant can navigate the virtual camera around the first three-dimensional model of the product.
9. The non-transitory, tangible computer-readable device of claim 7 , wherein the object is an advertisement.
10. The non-transitory, tangible computer-readable device of claim 7 , wherein the virtual environment is a building.
11. The non-transitory, tangible computer-readable device of claim 7 , further comprising:
transmitting a request, for the first mesh, from the second participant, wherein the receiving the first mesh occurs in response to the request.
12. The non-transitory, tangible computer-readable device of claim 7 , further comprising:
receiving a position and direction of the first participant in the three-dimensional virtual space;
receiving the video stream captured from a camera on a device of the first participant, the camera positioned to capture photographic images of the first participant;
wherein the rendering comprises rendering the virtual conference to include mapped three-dimensional model of the avatar positioned at the position and oriented at the direction.
13. A system for streaming video for a virtual video conference, comprising:
a processor coupled to a memory;
a network interface configured to receive (i) a three-dimensional model of a virtual environment, (ii) a first mesh representing a first three-dimensional model of an object, (iii) a video stream of a first participant of the virtual video conference, the video stream including a plurality of frames, wherein a second three-dimensional model represented by a second mesh is used to generate an avatar navigable by the first participant;
a mapper that maps respective frames from the video stream onto the second three-dimensional model to place the video stream on the avatar, wherein the second mesh is generated independently of the video stream; and
a renderer configured to, from a perspective of a virtual camera of a second participant, render for display for the second participant the mapped avatar and the first mesh representing the first three-dimensional model of the object within the virtual environment,
wherein demonstrating the object by simultaneously rendering the first three-dimensional model of the object in the virtual environment and presenting a second video stream of the first three-dimensional model of the object.
14. The system of claim 13 , wherein the object is a product, and wherein the second participant can navigate the virtual camera around the first three-dimensional model of the product.
15. The system of claim 13 , wherein the object is an advertisement.
16. The system of claim 13 , wherein the virtual environment is a building.