Systems and methods for modifying room characteristics for spatial audio rendering over headphones
An audio rendering system includes a processor that combines audio input signals with personalized spatial audio transfer functions having room responses. The personalized spatial audio transfer functions are selected from a database having a plurality of candidate transfer functions derived from in-ear microphone measurements for a plurality of individuals. Alternatively, the personalized transfer functions are derived from actual in-ear measurements of the listener. A room modification module allows the user to modify the personalized spatial audio transfer functions to substitute a different room or to modify the characteristics of the selected room without requiring additional in ear measurements. The module segments the selected transfer function into regions including one or more of direct; head and torso influenced; early reflection, and late reverberation regions. Extraction and modification operations are performed on one or more of the regions to alter the perceived sound.
1. A method for generating modified Binaural Room Impulse Reponses (BRIRs) comprising:
generating a first Binaural Room Impulse Response (BRIR) based on a matching operation applied to extracted biometric properties;
segmenting the first BRIR into at least two regions by determining at least two regions of four regions that include a direct region, an early reflections region, a head and torso influenced region, and a late reverberation region for the first BRIR;
applying deconvolution to the direct region of the first BRIR to remove first loudspeaker effects from the direct region and create a deconvolved direct region of the first BRIR, wherein the first loudspeaker effects comprise a measured loudspeaker impulse response of a first loudspeaker;
convolving an impulse response for a target loudspeaker with the deconvolved direct region of the first BRIR to at least partially generate at least one modified region, wherein the at least one modified region includes the deconvolved direct region of the first BRIR corresponding to different loudspeaker acoustic properties from the first loudspeaker, wherein the at least one modified region is further at least partially generated by a modification operation comprising adapting different perceived loudspeaker acoustic properties from the first BRIR; and
combining the at least one modified region and any unmodified regions of the first BRIR to form a modified BRIR, wherein the at least one modified region corresponds to a removal of head-related transfer function (HRTF) effects on a listener relating to a second BRIR to obtain a result of the removal of the HRTF effects on the listener and substituting the result of the removal of the HRTF effects on the listener into the first BRIR.
2. The method as recited in claim 1 , wherein the generating the at least one modified region includes at least one of truncation, altering the slope of the decay rate, windowing, smoothing, ramping, and full room swapping.
3. The method as recited in claim 1 , wherein the modified BRIR is intended to mimic an audio processing performed by the target loudspeaker different from the first loudspeaker used for the second BRIR and the at least one modified region is generated from a corresponding region culled from an impulse response for the target loudspeaker; and convolving the impulse response for the second loudspeaker with the deconvolved direct region of the first BRIR.
4. The method recited in claim 1 , wherein the modified BRIR is intended to mimic an audio processing performed by a target room different from a first room used for the first BRIR and the at least one modified region is generated from a corresponding region derived from an impulse response for the target room, and wherein the segmenting includes determining at least one of the early reflections region and the late reverberation region of the first BRIR, and further comprising applying changes to at least one of the early reflections region and the late reverberations region to reflect sound characteristics of the target room.
5. The method as recited in claim 4 , wherein the first loudspeaker effects are deconvolved from the first BRIR and further comprising convolving the impulse response for the target loudspeaker with the deconvolved direct region of the first BRIR for the first loudspeaker.
6. The method as recited in claim 1 , wherein the first BRIR is a BRIR for an individual generated by accessing a database comprising a candidate pool of BRIRs for a population of individuals, each of the BRIRs in the candidate pool indexed according to the extracted biometric properties.
7. A system for modifying room or speaker characteristics for spatial audio rendering over headphones including a processor performing the steps of:
generating a first BRIR based on a matching operation applied to extracted biometric properties;
receiving the first BRIR corresponding to a first loudspeaker in a first room;
segmenting the first BRIR into at least two regions by determining at least two regions of four regions that include a direct region, an early reflections region, a head and torso influenced region, and a late reverberation region for the first BRIR;
applying deconvolution to the direct region of the first BRIR to remove first loudspeaker effects from the direct region and create a deconvolved direct region of the first BRIR, wherein the first loudspeaker effects comprise a measured loudspeaker impulse response of a first loudspeaker;
convolving a target loudspeaker response with the deconvolved direct region of the first BRIR to at least partially generate at least one modified region, wherein the at least one modified region includes the deconvolved direct region of the first BRIR corresponding to different loudspeaker acoustic properties from the first loudspeaker, wherein the at least one modified region is further at least partially generated by a modification operation comprising adapting different perceived loudspeaker acoustic properties from the first BRIR; and
combining the at least one modified region and the unmodified regions of the first BRIR to form a modified BRIR, wherein the at least one modified region corresponds to changed sound attributes for a loudspeaker-room-listener interrelationship.
8. The system as recited in claim 7 , wherein the modified BRIR is intended to mimic changes in the sound attributes for a loudspeaker-room-listener interrelationship that are derived from at least one of changes in loudspeaker composition, loudspeaker distance to room walls, loudspeaker distance to listener, room size, room and of dimensions, room construction, or room furnishings.
9. The system as recited in claim 7 , wherein the modified BRIR is synthesized to simulate non-room environments and further comprising:
using the processor to segment the first BRIR into regions that include the direct region, the early reflections region, the head and torso influenced region, and the late reverberation region; and
using ray tracing to synthesize the new reverberation to remove the late reverberation region and the early reflections region to create the non-room environment with no early reflection and late reverberation.
10. A method for generating modified spatial audio transfer functions comprising:
generating a first spatial audio transfer function customized for an individual by accessing a spatial audio transfer function indexed according to extracted image based properties, wherein the generating is based on a matching operation applied to the extracted image based properties;
segmenting the first spatial audio transfer function into at least two regions by determining at least two regions of four regions that include a direct region, an early reflections region, a head and torso influenced region, and a late reverberation region for the first spatial audio transfer function; and
performing a modification operation on at least one of the at least two regions to at least partially generate at least one modified region, wherein the modification operation comprises adapting different perceived loudspeaker acoustic properties or room acoustic properties from the first spatial audio transfer function and performing deconvolution to the direct region of the first spatial audio transfer function to remove first loudspeaker effects from the direct region and create a deconvolved direct region of the first spatial audio transfer function, wherein the first loudspeaker effects comprise a measured loudspeaker impulse response of a first loudspeaker;
convolving a target loudspeaker response with the deconvolved direct region of the first spatial audio transfer function, wherein the at least one modified region is further at least partially generated by a modification operation comprising adapting different perceived loudspeaker acoustic properties from the first loudspeaker from the first spatial audio transfer function; and
combining the at least one modified region and any unmodified regions of the first spatial audio transfer function of the at least two regions to form a modified spatial audio transfer function that provides virtualization of the target loudspeaker having different acoustic properties than the loudspeaker used as a basis for the first spatial audio transfer function.