Audio encoding with compressed ambience

1. A method performed by an audio device, comprising:

sensing sound in a physical environment using a plurality of microphones to generate a plurality of microphone signals;

extracting clean speech from at least a portion of the plurality of microphone signals;

extracting ambience from at least a portion of the plurality of microphone signals; and

encoding, in a bit stream, the clean speech and the ambience by a) compressing the clean speech into an encoded speech signal at a first bit rate, and b) compressing the ambience into an encoded ambience signal at a second bit rate that is lower than the first bit rate.

2. The method of claim 1 , wherein the plurality of microphones are integral to the audio device; the audio device being one or more of the following: a head-worn device, a mobile device with display, a smart speaker, or a virtual reality headset; and the bit stream is transmitted to a second device through a communication protocol.

3. The method of claim 2 , wherein the audio device has a wireless transmitter and the communication protocol is a wireless communication protocol.

4. The method of claim 3 , further comprising

determining, based on the plurality of microphone signals, one or more acoustic parameters of the physical environment; and

including, in the bit stream, the one or more acoustic parameters, wherein the one or more acoustic parameters are applied, by a playback device, to the clean speech for playback.

5. The method of claim 4 , wherein the one or more acoustic parameters includes a reverberation decay time or a pattern of early reflections of the physical environment.

6. The method of claim 4 , wherein the one or more acoustic parameters includes one or more impulse responses of the physical environment, determined based on the plurality of microphone signals.

7. The method of claim 6 , wherein the one or more impulse responses includes a binaural room impulse response (BRIR).

8. The method of claim 4 , wherein the one or more acoustic parameters are determined based on a) one or more images of the physical environment, and b) measured reverberation of the physical environment based on the plurality of microphone signals.

9. The method of claim 1 , further comprising

generating, based on the microphone signals, one or more spatial parameters of a) the ambience, or b) the clean speech, the one or more spatial parameters defining spatial locations of the ambience or the clean speech in the physical environment; and

encoding the spatial parameters into the bit stream, the spatial parameters to be applied to the ambience or the clean speech by a playback device.

10. The method of claim 1 , wherein a bit rate of the encoded speech signal is 96 kB/sec or greater.

11. The method of claim 10 , wherein the second bit rate is less than one tenth the first bit rate.

12. The method of claim 1 , wherein the clean speech does not contain reverberant or ambient sound components.

13. The method of claim 1 , wherein extracting the clean speech includes applying dereverberation to the plurality of microphone signals.

14. The method of claim 1 , wherein extracting the clean speech includes forming a pick-up beam, from the plurality of microphone signals, focused at a speech source location in the physical environment.

15. The method of claim 1 , wherein extracting the ambience includes subtracting the clean speech from the microphone signals or steering a null pick-up beam at a speech source location in the physical environment.

16. The method of claim 1 , wherein

the bit stream further includes

a direction and a location associated with the speech, and

a visual representation of a speaker that is coordinated with the clean speech, and

the direction and the location are used by a playback device to spatialize the clean speech upon playback.

17. An audio device comprising:

a plurality of microphones to sense sound in a physical environment and generate a plurality of microphone signals; and

an audio processor configured to:

extract clean speech from at least a portion of the plurality of microphone signals,

extract ambience from at least a portion of the plurality of microphone signals, and

encode, in a bit stream, a) the clean speech in an encoded speech signal at a first compression level causing a first bit rate, and b) the ambience in an encoded ambience signal at a second compression level that is higher than the first compression level causing a second bit rate that is lower than the first bit rate.

18. The audio device of claim 17 wherein the plurality of microphones is integral to the audio device being a head-worn device, a mobile device with display, a smart speaker, or a virtual reality headset, wherein the audio device is to transmit the bit stream to a second device through a wireless communication protocol.

19. The audio device of claim 18 wherein the clean speech does not contain reverberant or ambient sound components.

20. The audio device of claim 19 , wherein the audio processor is further configured to:

determine, based on the plurality of microphone signals, one or more acoustic parameters of the physical environment;

generate, based on the microphone signals, one or more spatial parameters of a) the ambience, or b) the clean speech, the one or more spatial parameters defining spatial locations of the ambience or the clean speech in the physical environment; and

include, in the bit stream, the one or more acoustic parameters, and the one or more spatial parameters,

wherein the one or more acoustic parameters are to be applied, by a playback device, to the clean speech for playback, and

the spatial parameters are to be applied to the ambience or to the clean speech by the playback device for the playback.