Securing private audio in a virtual conference, and applications thereof
Disclosed herein is a computer-implemented method, system, device, and computer program product for securing private audio in a virtual conference. For each of the users in the virtual conference, a device of the speaking user or a server determines whether a respective user is able to hear the speaking user based on whether a respective sound volume at which the respective user is able to hear the speaking user exceeds a threshold amount. The speaking user's device or the server prevents the transmission of the audio stream to devices of the users determined not to be able to hear the speaking user.
1. A computer-implemented method for securing private audio for a virtual conference, comprising:
(a) receiving an audio stream captured from a microphone of a device of a speaking user, the microphone positioned to capture speech of the speaking user;
(b) from a perspective of a virtual camera of the speaking user, rendering for display to the speaking user a three-dimensional virtual space including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars;
for each user of the plurality of users:
(c) determining a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space;
(d) determining whether the user is able to hear the speaking user based on whether the sound volume exceeds a threshold amount; and
(e) preventing transmission of the audio stream to devices of each user of the plurality of users determined in (d) that is unable to hear the speaking user.
2. The computer-implemented method of claim 1 , further comprising:
(f) generating a list of each user of the plurality of users determined in (d) to be able to hear the speaking user; and
(g) transmitting the list generated in (f) and the audio stream to a server, wherein the server selectively transmits the audio stream to devices of the users in the list of users generated in (f).
3. The computer-implemented method of claim 1 , further comprising:
(f) transmitting a muffled version of the audio stream to the devices of each user of the plurality of users determined in (d) that is unable to hear the speaking user.
4. The computer-implemented method of claim 1 , further comprising:
(f) receiving position and direction information of at least one user of the plurality of users in the three-dimensional virtual space;
(g) receiving a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user;
(h) texture mapping the video stream onto a three-dimensional model of an avatar of the at least one user; and
(i) rendering, from the perspective of the virtual camera of the speaking user, for display to the speaking user, the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user.
5. The computer-implemented method of claim 1 , wherein an application downloaded on a web-browser renders the three-dimensional virtual space in (b).
6. The computer-implemented method of claim 1 , wherein the three-dimensional virtual space is segmented into a plurality of areas and each area in the plurality of areas comprises a wall transmission factor and a distance transmission factor.
7. The computer-implemented method of claim 6 , wherein the determining the sound volume in (c) is based on one or more of:
i) a distance between the virtual camera and the respective avatar, and the wall transmission factors of all areas between the virtual camera and the respective avatar; or
ii) the distance transmission factors of all areas between the virtual camera and the respective avatar.
8. A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for securing private audio for a virtual conference, comprising:
(a) receiving an audio stream captured from a microphone of a device of a speaking user, the microphone positioned to capture speech of the speaking user;
(b) from a perspective of a virtual camera of the speaking user, rendering for display to the speaking user a three-dimensional virtual space, including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars;
for each user of the plurality of users:
(c) determining a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space;
(d) determining whether a user is able to hear the speaking user based on whether the sound volume exceeds a threshold amount; and
(e) preventing transmission of the audio stream to devices of each user of the plurality of users determined in (d) that is unable to hear the speaking user.
9. The non-transitory, tangible computer-readable device of claim 8 , wherein the operations further comprise:
(f) generating a list of each user of the plurality of users determined in (d) to be able to hear the speaking user; and
(g) transmitting the list generated in (f) and the audio stream to a server, wherein the server selectively transmits the audio stream to devices of the users in the list of users generated in (f).
10. The non-transitory, tangible computer-readable device of claim 8 , wherein the operations further comprise:
(f) transmitting a muffled version of the audio stream to the devices of each user of the plurality of users determined in (d) that is unable to hear the speaking user.
11. The non-transitory, tangible computer-readable device of claim 8 , wherein the operations further comprise:
(f) receiving position and direction information of at least one user of the plurality of users in the three-dimensional virtual space;
(g) receiving a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user;
(h) texture mapping the video stream onto a three-dimensional model of an avatar of the at least one user; and
(i) rendering, from the perspective of the virtual camera of the speaking user, for display to the speaking user, the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user.
12. The non-transitory, tangible computer-readable device of claim 8 , wherein a web-application downloaded on a web-browser renders the three-dimensional virtual space in (b).
13. The non-transitory, tangible computer-readable device of claim 8 , wherein the three-dimensional virtual space is segmented into a plurality of areas and each area in the plurality of areas comprises a wall transmission factor and a distance transmission factor.
14. The non-transitory, tangible computer-readable device of claim 13 , wherein the determining of the sound volume in (c) is based on one or more of:
i) a distance between the virtual camera and the respective avatar, and the wall transmission factors of all areas between the virtual camera and the respective avatar; or
ii) the distance transmission factors of all areas between the virtual camera and the respective avatar.
15. A system for securing private audio for a virtual conference, comprising:
a memory;
a processor coupled to the memory, the processor configured to:
(a) receive an audio stream captured from a microphone of a device of a speaking user, the microphone positioned to capture speech of the speaking user;
(b) from a perspective of a virtual camera of the speaking user, render, using a web-application downloaded on a web-browser, for display to the speaking user, a three-dimensional virtual space, including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars;
for each user of the plurality of users:
(c) determine a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space;
(d) determine whether the user is able to hear the speaking user based on whether the sound volume exceeds a threshold amount; and
(e) prevent transmission of the audio stream to devices of each user of the plurality of users determined in (d) that is unable to hear the speaking user.
16. The system of claim 15 , wherein the processor is further configured to:
(f) generate a list of each user of the plurality of users determined in (d) to be able to hear the speaking user; and
(g) transmit the list generated in (f) and the audio stream to a server, wherein the server selectively transmits the audio stream to devices of the users in the list of users generated in (f).
17. The system of claim 15 , wherein the processor is further configured to:
(f) receive position and direction information of at least one user of the plurality of users in the three-dimensional virtual space;
(g) receive a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user;
(h) texture map the video stream onto a three-dimensional model of an avatar of the at least one user; and
(i) render, using the web-application downloaded on the web-browser, from the perspective of the virtual camera of the speaking user, for display to the speaking user the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user.
18. The system of claim 15 , wherein the three-dimensional virtual space is segmented into a plurality of areas and each areas in the plurality of areas comprises a wall transmission factor and a distance transmission factor, and
wherein the determining the sound volume in (c) is based on one or more of:
i) a distance between the virtual camera and the respective avatar, and the wall transmission factors of all areas between the virtual camera and the respective avatar; or
ii) the distance transmission factors of all areas between the virtual camera and the respective avatar.
19. A computer-implemented method for securing private audio for a virtual conference, comprising:
(a) receiving an audio stream captured from a microphone of a device of a speaking user, the microphone positioned to capture speech of the speaking user;
(b) from a perspective of a virtual camera of the speaking user, rendering for display to the speaking user a three-dimensional virtual space, including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars;
for each user of the plurality of users:
(c) determining whether the user is able to hear the speaking user based on information associated with the user;
and
(d) preventing transmission of the audio stream to devices of each user of the plurality of users determined in (c) that is unable to hear the speaking user.
20. The computer-implemented method of claim 19 , wherein determining that the user is able to hear the speaking user in (c) based on the information associated with the user indicating that:
the user is in a predetermined group of users able to hear the speaking user;
the user is of a predetermined user role; or
the user has predetermined security credentials.
21. The computer-implemented method of claim 19 , wherein determining that the user is not able to hear the speaking user in (c) based on the information associated with the user indicating that the user is actively blocked by the speaking user from receiving the audio stream.