Focusing on a portion of an audio scene for an audio signal
View Patent ↗Aspects of the invention provide methods, computer-readable media, and apparatuses for spatially manipulating sound that is played back to a listener over a set of output transducers, e.g., headphones. The listener can direct spatial attention to focus on a portion of an audio scene, analogous to a magnifying glass being used to pick out details in a picture. An input multi-channel audio signal that is generated by audio sources is obtained, and directional information is determined for each of the audio sources. The user provides a desired direction of spatial attention so that audio processing can focus on the desired direction and render a corresponding multi-channel audio signal to the user. A region of an audio scene is expanded around the desired direction while the audio scene is compressed in another portion of the audio scene.
1. A method comprising:
obtaining a single-channel or multi-channel input audio signal having a plurality of audio sources;
generating, with a processor, a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;
obtaining at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and
expanding a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes:
for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, moving the sound source in the modified sound field according to a re-mapping function; and
for any sound source centered in the indicated at least one direction of spatial attention in the sound field, maintaining the sound source centered in the indicated at least one direction of spatial attention in the modified sound field.
2. The method of claim 1 , further comprising:
compressing a second region of the sound field to produce the modified sound field.
3. The method of claim 1 , further comprising:
re-mapping an azimuth value of each sound source in the first region and not centered in the indicated at least one direction of spatial attention to a new azimuth value in the modified sound field.
4. The method of claim 1 , further comprising:
utilizing a remapping function to re-map each azimuth value, wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.
5. The method of claim 1 , further comprising:
preserving an overall loudness of the plurality of audio sources when moving one or more of the plurality of audio sources in the modified sound field of the output audio signal.
6. The method of claim 1 , further comprising:
amplifying one or more of the audio sources positioned within the first region of the sound field.
7. The method of claim 1 , the output audio signal comprising a binaural audio signal.
8. The method of claim 1 , further comprising:
obtaining the at least one parameter indicating the at least one direction of spatial attention from a headtracker configured to be fastened to the user.
9. An apparatus comprising:
at least one processor;
and memory having computer executable instructions stored thereon, that when executed, cause the apparatus to:
obtain a single-channel or multi-channel input audio signal having a plurality of audio sources;
generate a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;
obtain at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and
expand a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes:
for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, move the sound source in the modified sound field according to a re-mapping function; and
for any sound source centered in the indicated at least one direction of spatial attention in the sound field, maintain the sound source centered in the indicated at least one direction of spatial attention in the modified sound field.
10. The apparatus of claim 9 , wherein the computer executable instructions, when executed, cause the apparatus to:
compress a second region of the sound field to produce the modified sound field.
11. The apparatus of claim 9 , wherein the computer executable instructions, when executed, cause the apparatus to:
re-map an azimuth value of each sound source in the first region and not centered in the indicated at least one direction of spatial attention to a new azimuth value in the modified sound field.
12. The apparatus of claim 11 , wherein the computer executable instructions, when executed, cause the apparatus to:
utilize a re-mapping function to re-map each azimuth value, wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.
13. A method comprising:
obtaining a single-channel or multi-channel input audio signal having a plurality of audio sources;
generating a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;
obtaining at least one parameter from a user control led in gut device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and
expanding a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes:
for each sound source in the first region, modifying in the modified sound field an azimuth angle between the sound source and the indicated at least one direction of spatial attention according to a re-mapping function having a non-zero derivative at the indicated at least one direction of spatial attention.
14. The method of claim 13 , further comprising:
compressing a second region of the sound field to produce the modified sound field.
15. The method of claim 13 , wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.
16. An apparatus comprising:
at least one processor;
and memory having computer executable instructions stored thereon, that when executed, cause the apparatus to:
obtain a single-channel or multi-channel audio signal having a plurality of audio sources;
generate a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;
obtain at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and
expand a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes:
for each sound source in the first region, modifying in the modified sound field an azimuth angle between the sound source and the indicated at least one direction of spatial attention according to a re-mapping function having a non-zero derivative at the indicated at least one direction of spatial attention.
17. The apparatus of claim 16 , wherein the computer executable instructions, when executed, cause the apparatus to:
compress a second region of the sound field to produce the modified sound field.
18. The method of claim 1 , further comprising:
determining directional information for each of the plurality of audio sources in the input audio signal; and
generating the output audio signal by positioning the plurality of audio sources in the sound field based on the directional information.
19. The apparatus of claim 9 , wherein the computer executable instructions, when executed, cause the apparatus to:
determine directional information for each of the plurality of audio sources in the input audio signal; and
generate the output audio signal by positioning the plurality of audio sources in the sound field based on directional information.
20. The method of claim 1 , further comprising:
rendering the output audio signal from one or more speakers.
21. The apparatus of claim 9 , wherein the computer executable instructions, when executed, cause the apparatus to:
render the output audio signal from one or more speakers.
22. The method of claim 1 , further comprising:
for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, moving the sound source in the modified sound field away from the at least one direction of spatial attention according to the re-mapping function.