Statistical provisioning of perceptual audio cues to enhance speech
Methods, systems, and storage media for generating spatial audio are disclosed. Exemplary implementations may: receive audio data from a plurality of sources; for each source of the plurality of sources, calculate a pitch similarity for the audio data; for each source of the plurality of sources, calculate an interaural level difference based at least in part on an attenuation level and/or a dynamic range of interaural time differences of the audio data; for each source of the plurality of sources, determine background audio from the audio data; for each source of the plurality of sources, attenuate the background audio of the audio data; generate spatial audio based at least in part on the audio data and the background audio; and cause output of the audio data and the background audio through an audio source.
1 . A computer-implemented method for generating spatial audio, comprising:
receiving audio data from a plurality of sources;
for each source of the plurality of sources:
calculating a pitch similarity for the audio data, the pitch similarity determined in real time to dynamically adjust perceptual audio cues, wherein the perceptual audio cues comprise one or more of whispered backgrounds, time-dilated vowels, and enhanced sound onsets; and
calculating an interaural level difference based at least in part on at least one of an attenuation level or a dynamic range of interaural time differences of the audio data;
generating spatial audio based at least in part on at least one of the pitch similarity of the audio data or the interaural level difference of the audio data; and
causing output of audio, through one or more speakers, based at least in part on the generated spatial audio.
2 . The method of claim 1 , wherein calculating the pitch similarity comprises filtering the audio data between 200 Hz and 2500 Hz.
3 . The method of claim 1 , further comprising, for each source of the plurality of sources:
determining background audio from the audio data; and
attenuating the background audio of the audio data, wherein the attenuating comprises filtering the background audio into contiguous narrow bands.
4 . The method of claim 3 , wherein the attenuated background audio comprises whispered audio.
5 . The method of claim 1 , wherein the time-dilated vowels are configured by dilating vowel sounds of the audio data through time dilation.
6 . The method of claim 5 , wherein a time dilation ratio is around 10%.
7 . The method of claim 1 , wherein the enhanced sound onsets are configured for each source of the plurality of sources by:
increasing a steepness of an onset slope of subsequent sound by widening a gap between sound samples of the audio data.
8 . The method of claim 1 , wherein the plurality of sources includes representations of different entities within an artificial reality environment.
9 . The method of claim 1 , wherein the audio data is received through microphones of users' head-mounted displays configured for artificial reality experiences.
10 . The method of claim 1 , wherein calculating the interaural level difference includes determining a relative difference between audio signals impinging on a right ear of a user and on a left ear of the user, and wherein calculating the interaural time differences includes determining a relative difference between arrival times of an audio signal at the right ear and the left ear of the user.
11 . A system configured to generate spatial audio, the system comprising: one or more hardware processors configured by machine-readable instructions to:
receive audio data from a plurality of sources;
for each source of the plurality of sources:
calculate a pitch similarity for the audio data, the pitch similarity determined in real time to dynamically adjust perceptual audio cues, wherein the perceptual audio cues comprise one or more of whispered backgrounds, time-dilated vowels, and enhanced sound onsets; and
calculate an interaural level difference based at least in part on an attenuation level or a dynamic range of interaural time differences of the audio data;
generate spatial audio based at least in part on at least one of the pitch similarity of the audio data or the interaural level difference of the audio data; and
cause output of audio, through one or more speakers, based at least in part on the generated spatial audio.
12 . The system of claim 11 , wherein the one or more hardware processors are further configured by machine-readable instructions to, for each source of the plurality of sources:
determine background audio from the audio data; and
attenuate the background audio of the audio data, wherein the attenuating comprises filtering the background audio into contiguous narrow bands.
13 . The system of claim 12 , wherein the attenuated background audio comprises whispered audio.
14 . The system of claim 11 , wherein calculating the pitch similarity comprises filtering the audio data between 200 Hz and 2500 Hz.
15 . The system of claim 11 , wherein the one or more hardware processors are further configured by machine-readable instructions to dilate vowel sounds of the audio data through time dilation to create the time-dilated vowels.
16 . The system of claim 15 , wherein a time dilation ratio is around 10%.
17 . The system of claim 11 , wherein the one or more hardware processors are further configured by machine-readable instructions to, for each source of the plurality of sources:
increase a steepness of an onset slope of subsequent sound by widening a gap between sound samples of the audio data to create the enhanced sound onsets.
18 . The system of claim 11 , wherein the plurality of sources includes representations of different entities within an artificial reality environment.
19 . The system of claim 11 , wherein the audio data is received through microphones of users' head-mounted displays configured for artificial reality experiences.
20 . A non-transient computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving audio data from a plurality sources;
for each source of the plurality of sources:
calculating a pitch similarity for the audio data, the pitch similarity determined in real time to dynamically adjust perceptual audio cues, wherein the perceptual audio cues comprise one or more of whispered backgrounds, time-dilated vowels, and enhanced sound onsets; and
calculating an interaural level difference based at least in part on at least one of an attenuation level or a dynamic range of interaural time differences of the audio data;
generating spatial audio based at least in part on at least one of the pitch similarity of the audio data or the interaural level difference of the audio data; and
causing output of audio, through one or more speakers, based at least in part on the generated spatial audio.