Presenter mode in a three-dimensional virtual conference space, and applications thereof
Disclosed herein is a web-based videoconference system that allows for video avatars to navigate within the virtual environment. The system has a presented mode that allows for a presentation stream to be texture mapped to a presenter screen situated within the virtual environment. The relative left-right sound is adjusted to provide sense of an avatar's position in a virtual space. The sound is further adjusted based on the area where the avatar is located and where the virtual camera is located. Video stream quality is adjusted based on relative position in a virtual space. Three-dimensional modeling is available inside the virtual video conferencing environment.
1. A computer-implemented method for presenting in a virtual conference including a plurality of participants, comprising:
receiving data specifying a three-dimensional virtual space;
receiving a position and direction in the three-dimensional virtual space, the position and direction input by a first participant of the plurality of participants to the virtual conference;
receiving a video stream captured from a camera on a device of the first participant, the camera positioned to capture photographic images of the first participant;
mapping the video stream onto a three-dimensional model of an avatar;
receiving a presentation stream from the device of the first participant;
mapping the presentation stream onto a three-dimensional model of a presentation screen; and
from a perspective of a virtual camera of a second participant of the plurality of participants, rendering for display to the second participant the three-dimensional virtual space with the mapped avatar and the mapped presentation screen.
2. The method of claim 1 , further comprising:
receiving an audio stream captured synchronously with the presentation stream and from a microphone of the device of the first participant, the microphone positioned to capture speech of the first participant; and
outputting the audio stream to play to the second participant synchronously with display of the presentation stream within the three-dimensional virtual space.
3. The method of claim 1 , further comprising:
receiving a position in the three-dimensional virtual space of a third participant in the plurality of participants;
receiving an audio stream from a microphone of a device of the third participant, the microphone positioned to capture speech of the third participant; and
adjusting the audio stream to provide a sense of the received position in the three-dimensional virtual space of the third participant relative to a position of the virtual camera,
wherein the rendering comprises rendering for display to the second participant the three-dimensional virtual space with an avatar, for the third participant, at the received position.
4. The method of claim 1 , further comprising:
receiving a position in the three-dimensional virtual space of the first participant;
receiving an audio stream from a microphone of a device of the first participant, the microphone positioned to capture speech of the first participant;
adjusting the audio stream to provide a sense of the received position in the three-dimensional virtual space of the first participant relative to a position of the virtual camera;
rendering for display to the second participant the three-dimensional virtual space with an avatar, for the third participant, at the received position; and
when a presentation mode is entered, adjusting the audio stream to provide a sense of a position of the mapped presentation screen relative to a position of the virtual camera.
5. The method of claim 1 , wherein the presentation stream is video of the first participant.
6. The method of claim 1 , wherein the presentation stream is screen share of the first participant.
7. The method of claim 1 , wherein the mapping the video stream comprises mapping respective frames of the video stream onto the three-dimensional model of the avatar to present moving images of a face of the first participant on the avatar.
8. The method of claim 7 , wherein the avatar includes a surface and wherein the mapping comprises mapping the respective frames onto the surface.
9. The method of claim 8 , wherein the rendering comprises rendering the mapped avatar at the position and direction in the three-dimensional virtual space, wherein the first participant is able to change the position and direction of the mapped avatar within the rendered three-dimensional virtual space based on a change of the position and the direction input by the first participant.
10. The method of claim 8 , wherein the rendering comprises rendering such that the avatar is located at the position in the three-dimensional virtual space and the surface is oriented in the direction in the three-dimensional virtual space, further comprising:
receiving a new direction in the three-dimensional virtual space, the new direction input by the first participant;
from the perspective of the virtual camera of the second participant, re-rendering for display to the second participant the three-dimensional virtual space such that the surface of the texture-mapped avatar is oriented in the new direction.
11. The method of claim 10 , wherein, when the new direction is input the first participant, a virtual camera of the first participant is changed according to the new direction, the virtual camera of the first participant specifying how the three-dimensional virtual space is rendered for display to the first participant.
12. A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for presenting in a virtual conference including a plurality of participants, the operations comprising:
receiving data specifying a three-dimensional virtual space;
receiving a position and direction in the three-dimensional virtual space, the position and direction input by first participant of the plurality of participants to the virtual conference;
receiving a video stream captured from a camera on a device of the first participant, the camera positioned to capture photographic images of the first participant;
mapping the video stream onto a three-dimensional model of an avatar;
receiving a presentation stream from a device of the first participant;
mapping the presentation stream onto a three-dimensional model of a presentation screen; and
from a perspective of a virtual camera of a second participant of the plurality of participants, rendering for display to the second participant the three-dimensional virtual space with the mapped avatar and the mapped presentation screen.
13. The device of claim 12 , the operations further comprising:
receiving an audio stream captured synchronously with the presentation stream and from a microphone of the device of the first participant, the microphone positioned to capture speech of the first participant; and
outputting the audio stream to play to the second participant synchronously with display of the presentation stream within the three-dimensional virtual space.
14. The device of claim 12 , the operations further comprising:
receiving a position in the three-dimensional virtual space of a third participant in the plurality of participants;
receiving an audio stream from a microphone of a device of the third participant, the microphone positioned to capture speech of the third participant; and
adjusting the audio stream to provide a sense of the received position in the three-dimensional virtual space of the third participant relative to a position of the virtual camera,
wherein the rendering comprises rendering for display to the second participant the three-dimensional virtual space with an avatar, for the third participant, at the received position.
15. The device of claim 12 , the operations further comprising:
receiving a position in the three-dimensional virtual space of the first participant;
receiving an audio stream from a microphone of a device of the first participant, the microphone positioned to capture speech of the first participant;
adjusting the audio stream to provide a sense of the received position in the three-dimensional virtual space of the first participant relative to a position of the virtual camera;
rendering for display to the second participant the three-dimensional virtual space with an avatar, for the third participant, at the received position; and
when a presentation mode is entered, adjusting the audio stream to provide a sense of a position of the mapped presentation screen relative to a position of the virtual camera.
16. The device of claim 12 , wherein the presentation stream is video of the first participant.
17. The device of claim 12 , wherein the presentation stream is a screen share of the first participant.
18. A system for presenting in a virtual conference including a plurality of participants, comprising:
a processor coupled to a memory;
a network interface configured to receive: (i) data specifying a three-dimensional virtual space, (ii) a position and direction in the three-dimensional virtual space, the position and direction input by a first participant of the plurality of participants to the virtual conference, (iii) a video stream captured from a camera on a device of the first participant, the camera positioned to capture photographic images of the first participant, and (iv) a presentation stream from a device of the first participant;
a mapper, implemented on the processor, configured to map the video stream onto a three-dimensional model of an avatar, and map the presentation stream onto a three-dimensional model of a presentation screen; and
a renderer, implemented on the processor, configured to render for display to the second participant, from a perspective of a virtual camera of a second participant of the plurality of participants, the three-dimensional virtual space with the mapped avatar and the mapped presentation screen.
19. The system of claim 18 , wherein the presentation stream is video of the first participant.
20. The system of claim 18 , wherein the presentation stream is a screen share of the first participant.
21. A computer-implemented method for presenting in a virtual conference including a plurality of participants, comprising:
receiving, from a first device of a first participant of the plurality of participants to the virtual conference, (i) a position and direction in the three-dimensional virtual space, the position and direction input by the first participant, (ii) a video stream captured from a camera on the first device, the camera positioned to capture photographic images of the first participant, and (iii) a presentation stream; and
transmitting, to a second device of a second participant of the plurality of participants, the presentation stream, wherein the second device is configured (i) to map the presentation stream onto a three-dimensional model of a presentation screen, (ii) to map the video stream to an avatar, and, (iii) from a perspective of a virtual camera of the second participant of the plurality of participants, to render for display to the second participant a three-dimensional virtual space with the mapped presentation screen space and the mapped three-dimensional model of the avatar located at the position and oriented at the direction.
22. The method of claim 21 , wherein the presentation stream is video of the first participant.
23. The method of claim 21 , wherein the presentation stream is screen share of the first participant.
24. The method of claim 21 , further comprising:
transmitting, to the second device, a web application with executable code specifying how the second device is to map and render the presentation screen.