Reverberation generation for headphone virtualization
The present disclosure relates to reverberation generation for headphone virtualization. A method of generating one or more components of a binaural room impulse response (BRIR) for headphone virtualization is described. In the method, directionally-controlled reflections are generated, wherein directionally-controlled reflections impart a desired perceptual cue to an audio input signal corresponding to a sound source location. Then at least the generated reflections are combined to obtain the one or more components of the BRIR. Corresponding system and computer program products are described as well.
1. A method of generating an audio signal, comprising:
generating, by one or more processors using a directional pattern, directionally-controlled reflections, wherein:
the directional pattern corresponds to a perceptual cue and describes how directions of arrival of the directionally-controlled reflections oscillate in relation to a direction of a sound source location as a function of time, and
the directionally-controlled reflections impart the perceptual cue to an audio input signal corresponding to the sound source location;
obtaining one or more components of a binaural room impulse response (BRIR) by combining the directionally-controlled reflections; and
generating the audio signal for a playback device based on the BRIR.
2. The method of claim 1 , wherein the directional pattern has a wobble shape in which the directions of arrival of the directionally-controlled reflections change away from the direction of the sound source location and oscillates back and forth as the function of time.
3. The method of claim 1 , wherein the audio signal includes a binaural signal for the playback device.
4. The method of claim 1 , wherein the directionally-controlled reflections have directions of arrival in which an illusion of a virtual sound source at a given location in space is enhanced.
5. The method of claim 1 , wherein the directions of arrival of the directionally-controlled reflections comprise a stochastic diffuse component within an azimuths range, and wherein at least one of a wobble shape of the directional pattern or the stochastic diffuse component is selected based on a direction of the sound source location.
6. The method of claim 1 , wherein generating directionally-controlled reflections comprises:
determining respective occurrence time points of reflections stochastically under a predetermined echo density distribution constraint;
determining desired directions of the reflections based on respective occurrence time points and the directional pattern;
determining amplitudes of the reflections at the respective occurrence time points stochastically; and
creating the reflections with the desired directions and the determined amplitudes at the respective occurrence time points.
7. The method of claim 6 , wherein creating the directionally-controlled reflections comprises:
determining one or more head-related transfer functions (HRTFs) based on the directions at the respective occurrence time points; and
modifying the one or more HRTFs based on amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.
8. The method of claim 6 , wherein creating the directionally-controlled reflections comprises:
determining HRTFs based on the desired directions at the respective occurrence time points and a predetermined spherical head model; and
modifying the HRTFs based on the amplitudes of the reflections at the respective occurrence time points so as to obtain the reflections at the respective occurrence time points.
9. The method of claim 6 , wherein creating the directionally-controlled reflections comprises:
generating impulse responses for two ears based on desired directions and determined amplitudes at the respective occurrence time points and based on broadband interaural time difference and interaural level difference of a predetermined spherical head model.
10. The method of claim 9 , wherein creating the directionally-controlled reflections further comprises:
filtering the impulse responses for two ears through all-pass filters to obtain a diffusion and decorrelation.
11. The method of claim 1 , wherein the method is operated in a feedback delay network, and wherein generating reflections comprises filtering the audio input signal through HRTFs.
12. The method of claim 11 , wherein the audio input signal is delayed by delay lines before it is filtered by the HRTFs.
13. The method of claim 11 , wherein the audio input signal is filtered before signals fed back through at least one feedback matrix are added.
14. The method of claim 11 , wherein the audio input signal is filtered by the HRTFs in parallel with the audio input signal being inputted into the feedback delay network, and wherein output signals from the feedback delay network and from the HRTFs are mixed to obtain reverberation for headphone virtualization.
15. The method of claim 11 , wherein for multiple audio channels or objects, an input audio signal for each of the multiple audio channels or objects is separately filtered by the HRTFs.
16. The method of claim 11 , wherein for multiple audio channels or objects, input audio signals for the multiple audio channels or objects are downmixed and analyzed to obtain an audio mixture signal with a dominant source direction, which is taken as the audio input signal.
17. The method of claim 1 , further comprising performing operations comprising at least one of:
repeating generating the directionally-controlled reflections to obtain a plurality of groups of reflections and selecting one of the plurality of groups of reflections having an optimal reflection characteristic as the reflections for the audio input signal; or
repeating generating the directionally-controlled reflections till a particular reflection characteristic is obtained.
18. A system comprising:
one or more processors; and
a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations of generating an audio signal, the operations comprising:
generating, by one or more processors using a directional pattern, directionally-controlled reflections, wherein:
the directional pattern corresponds to a perceptual cue and describes how directions of arrival of the directionally-controlled reflections oscillate in relation to a direction of a sound source location as a function of time, and
the directionally-controlled reflections impart the perceptual cue to an audio input signal corresponding to the sound source location;
obtaining one or more components of a binaural room impulse response (BRIR) by combining the directionally-controlled reflections; and
generating the audio signal for a playback device based one the BRIR.
19. The system of claim 18 , wherein the directional pattern has a wobble shape in which the directions of arrival of the directionally-controlled reflections change away from the direction of the sound source location and oscillates back and forth as the function of time.
20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations of generating an audio signal, the operations comprising:
generating, by one or more processors using a directional pattern, directionally-controlled reflections, wherein:
the directional pattern corresponds to a perceptual cue and describes how directions of arrival of the directionally-controlled reflections oscillate in relation to a direction of a sound source location as a function of time, and
the directionally-controlled reflections impart the perceptual cue to an audio input signal corresponding to the sound source location;
obtaining one or more components of a binaural room impulse response (BRIR) by combining the directionally-controlled reflections; and
generating the audio signal for a playback device based on the BRIR.