Spatial audio and avatar control at headset using audio signals

1. A headset comprising:

a display;

a transducer assembly;

a transceiver configured to:

receive an audio signal from an audio system located remotely from a user of the headset in a local area, the audio signal generated at the audio system based in part on a location of the user within a virtual representation of the local area relative to a virtual representation of a human sound source within the virtual representation of the local area, and

receive information about a facial expression for a portion of a face of the human sound source, the information about the facial expression generated at the audio system based at least in part on identified sounds from the human sound source responsive at least in part to the portion of the face not being within a field of view of at least one imaging device of the audio system; and

a controller configured to:

instruct the transducer assembly based on the received audio signal to generate sound for presentation to the user within the virtual representation of the local area, and

instruct the display to present the facial expression to the user in sync with the presented sound, based on the received information about the facial expression.

2. The headset of claim 1 , wherein the facial expression is predicted responsive to a virtual rendering of the portion of the face being within a threshold angle of a field of view of the user.

3. The headset of claim 1 , wherein the facial expression comprises a lips movement of the virtual representation of the human sound source.

4. The headset of claim 1 , wherein the facial expression is predicted by applying a machine learning algorithm on the identified sounds from the human sound source.

5. The headset of claim 1 , further comprising:

at least one acoustic sensor configured to detect the sounds from the human sound source, wherein

the transceiver is further configured to transmit information about the detected sounds to the audio system.

6. The headset of claim 1 , wherein:

the transceiver is further configured to receive, from the audio system, a second audio signal generated at the audio system based in part on the location of the user within the virtual representation of the local area relative to a virtual representation of a second human sound source within the virtual representation of the local area; and

the controller is further configured to instruct the transducer assembly to generate a second sound for presentation to the user within the virtual representation of the local area, based on the received second audio signal.

7. The headset of claim 1 , wherein the sounds from the human sound source are identified by performing beam-steering type processing of sounds detected within the local area based on location information of the human sound source in the local area.

8. The headset of claim 1 , wherein the headset is communicatively coupled with a plurality of base stations in the local area including a plurality of arrays of acoustic sensors configured to detect at least a portion of sounds within the local area and a plurality of imaging devices configured to image at least a portion of the local area.

9. The headset of claim 8 , wherein:

the transceiver is further configured to:

receive audio information associated with the detected portion of sounds, and

receive video information associated with the imaged portion of the local area; and

the controller is further configured to:

instruct the transducer assembly to generate an audio stream for presentation to the user within the virtual representation of the local area based on the received audio information, and

instruct the display to present the video information to the user in sync with the audio stream.

10. The headset of claim 1 , wherein the transducer assembly is configured to:

generate an audio stream based on sound in the local area including a voice signal of the user; and

present the audio stream to the user within the virtual representation of the local area in synch with the presented facial expression.

11. A method performed by a headset, the method comprising:

receiving an audio signal from an audio system located remotely from a user of the headset in a local area, the audio signal generated at the audio system based in part on a location of the user within a virtual representation of the local area relative to a virtual representation of a human sound source within the virtual representation of the local area;

receiving information about a facial expression for a portion of a face of the human sound source, the information about the facial expression generated at the audio system based at least in part on identified sounds from the human sound source responsive at least in part to the portion of the face not being within a field of view of at least one imaging device of the audio system;

generating sound for presentation to the user within the virtual representation of the local area based on the received audio signal; and

presenting the facial expression to the user in sync with the presented sound based on the received information about the facial expression.

12. The method of claim 11 , wherein the facial expression is predicted responsive to a virtual rendering of the portion of the face being within a threshold angle of a field of view of the user.

13. The method of claim 11 , wherein the facial expression is predicted by applying a machine learning algorithm on the identified sounds from the human sound source.

14. The method of claim 11 , further comprising:

detecting the sounds from the human sound source; and

transmitting, via a network, information about the detected sounds to the audio system.

15. The method of claim 11 , further comprising:

receiving, from the audio system via a network, a second audio signal generated at the audio system based in part on the location of the user within the virtual representation of the local area relative to a virtual representation of a second human sound source within the virtual representation of the local area; and

generating a second sound for presentation to the user within the virtual representation of the local area based on the received second audio signal.

16. The method of claim 11 , further comprising:

receiving audio information associated with sounds within the local area detected by a plurality of arrays of acoustic sensors of a plurality of base stations in the local area;

receiving video information associated with at least a portion of the local area imaged by a plurality of imaging devices of the plurality of base stations; and

presenting to the user the audio information in sync with the video information.

17. The method of claim 11 , further comprising:

generating an audio stream based on sound in the local area including a voice signal of the user; and

presenting the audio stream to the user within the virtual representation of the local area in synch with the presented facial expression.

18. A computer program product comprising a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to:

receive an audio signal from an audio system located remotely from a user of a headset in a local area, the audio signal generated at the audio system based in part on a location of the user within a virtual representation of the local area relative to a virtual representation of a human sound source within the virtual representation of the local area;

receive information about a facial expression for a portion of a face of the human sound source, the information about the facial expression generated at the audio system based at least in part on identified sounds from the human sound source responsive at least in part to the portion of the face not being within a field of view of at least one imaging device of the audio system;

instruct the headset based on the received audio signal to generate sound for presentation to the user within the virtual representation of the local area; and

instruct the headset to present the facial expression to the user in sync with the presented sound, based on the received information about the facial expression.

19. The computer program product of claim 18 , wherein, the instructions further cause the one or more processors to:

receive audio information associated with sounds within the local area detected by a plurality of arrays of acoustic sensors of a plurality of base stations in the local area;

receive video information associated with at least a portion of the local area imaged by a plurality of imaging devices of the plurality of base stations; and

instruct the headset to present to the user the audio information in sync with the video information.

20. The computer program product of claim 18 , wherein the instructions further cause the one or more processors to:

generate an audio stream based on sound in the local area including a voice signal of the user; and

instruct the headset to present the audio stream to the user within the virtual representation of the local area in synch with the presented facial expression.