IP Library › Granted Patent US 12,242,972
Granted Patent B2
US 12,242,972 · App. 18/521,850 · Granted Mar 4, 2025

Methods and systems for generating acoustic impulse responses

Inventors: Finnur Pind (Reykjavík, IS); Jesper Pedersen (Garðabær, IS)
Assignee: TREBLE TECHNOLOGIES
G06N3/094G06N3/0455H04S7/302H04S7/303H04S7/305H04S7/307H04S2400/11H04S2400/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,972
App. No.
18/521,850
Granted
Mar 4, 2025
Kind
B2
Abstract

A method for generating an impulse response for a listening point in a room includes: receiving a 3D model of the room, the position of at least one sound source in the 3D model, and acoustic properties of at least one boundary in the 3D model; using a wave based solver for determining a wave based impulse response of a wave based propagation of an impulse emitted at the at least one sound source in the 3D model and received at the listening point within a first acoustic frequency range; using a geometrical acoustics based solver for determining a geometrical impulse response of the ray based propagation of an impulse emitted at the at least one sound source in the 3D model and received at the listening point within a second acoustic frequency range; and generating the impulse response by merging the wave impulse response and the geometrical impulse response.

Claims (30)

1. A computer implemented method for generating an impulse response for a listening point in a room, the method comprising:

receiving a 3D model of the room, a position of at least one directive sound source for emitting sound in a defined direction in the 3D model of the room, and acoustic properties of at least one boundary in the 3D model of the room;

determining, using a time-domain discontinuous Galerkin finite element method (DGFEM), a plurality of wave-based impulse responses of a wave-based propagation of a plurality of impulses emitted at the at least one directive sound source for emitting sound in a defined direction in the 3D model of the room and received at the listening point within a first acoustic frequency range, wherein the at least one directive sound source is modelled using spherical harmonics functions, wherein each of the plurality of impulses matches a directivity pattern associated with at least one spherical harmonics function;

generating a directional wave-based impulse response of the wave-based propagation of the plurality of impulses emitted at the at least one directive sound source by combining the plurality of wave-based impulse responses;

determining, using a geometrical acoustics-based solver, a geometrical impulse response of a ray-based propagation of an impulse emitted at the at least one directive sound source for emitting sound in a defined direction in the 3D model of the room and received at the listening point within a second acoustic frequency range; and

generating the impulse response by merging the directional wave-based impulse response and the geometrical impulse response.

2. The computer implemented method according to claim 1 , wherein the first and second acoustic frequency ranges partly overlap.

3. The computer implemented method according to claim 1 , wherein the computer implemented method further comprises generating a mesh model of the 3D model of the room, wherein the mesh model is a 3D curvilinear mesh model.

4. The computer implemented method according to claim 1 , further comprising adjusting a power level of the at least one sound source such that a sound level received at a predetermined distance from the at least one sound source is the same in the time-domain DGFEM and in the geometrical acoustic solver.

5. The computer implemented method according to claim 1 , wherein an upper frequency of the first acoustic frequency range and a lower frequency of the second frequency range overlap at a transition frequency.

6. The computer implemented method according to claim 5 , wherein merging the wave-based impulse response and the geometrical impulse response comprises applying a low pass filter to the wave-based impulse response.

7. The computer implemented method according to claim 5 , wherein merging the wave-based impulse response and the geometrical impulse response comprises applying a high pass filter to the geometrical impulse response.

8. The computer implemented method according to claim 7 , wherein at least one of the low pass filter and the high pass filter comprises a cut off frequency at the transition frequency.

9. The computer implemented method according to claim 1 , wherein using at least one of the time-domain DGFEM and the geometrical acoustic solver comprises extracting at least one of one or more wave impulse responses and one or more geometrical impulse responses based on a simulation of a propagation of sound and wherein the one or more wave impulse responses are spatial impulse responses.

10. The computer implemented method according to claim 9 , wherein the one or more spatial impulse responses comprise a plurality of single channel impulse responses, wherein each one of the plurality of single channel impulse responses corresponds to the wave impulse response from a specific direction or angle at a same listening point.

11. The computer implemented method according to claim 1 , wherein a spherical receiver array is arranged around the listening point, wherein the spherical receiver array comprises a plurality of receivers.

12. The computer implemented method according to claim 11 , wherein the spherical receiver array is an open spherical array of cardioid receivers.

13. The computer implemented method according to claim 11 , wherein the spherical receiver array comprises at least 4 receivers.

14. The computer implemented method according to claim 11 , wherein a number of receivers is determined based on the maximum truncation order N, such that the number of receivers is higher or equal to (N+1) 2 .

15. The computer implemented method according to claim 1 , further comprising convolving the generated impulse response with a base audio signal such that a convolved audio signal is generated.

16. The computer implemented method according to claim 1 , further comprising rendering a base audio signal by convolving the base audio signal with the generated impulse response, thereby creating a rendered audio signal.

17. The computer implemented method according to claim 16 , wherein the base audio signal is selected from the group consisting of speech, music, an environmental sound, an impulse sound, and combinations thereof.

18. The computer implemented method according to claim 16 , wherein the rendered audio signal provides an audio rendering of the base audio signal in the 3D model of the room at the listening point.

19. A system for generating an impulse response for a listening point in a room, the system comprising:

a computer system having a processor coupled to a memory, the processor configured to:

receive a 3D model of the room, a position of at least one directive sound source for emitting sound in a defined direction in the 3D model of the room, and acoustic properties of at least one boundary in the 3D model of the room;

determine, using a time-domain discontinuous Galerkin finite element method (DGFEM), a plurality of wave-based impulse responses of a wave-based propagation of a plurality of impulses emitted at the at least one directive sound source for emitting sound in a defined direction in the 3D model of the room, wherein the at least one directive sound source is modelled using spherical harmonics functions, and received at the listening point within a first acoustic frequency range, wherein each of the plurality of impulses matches a directivity pattern associated with at least one spherical harmonics function;

generate a directional wave-based impulse response of the wave-based propagation of the plurality of impulses emitted at the at least one directive sound source by combining the plurality of wave-based impulse responses;

determine, using a geometrical acoustics-based solver, a geometrical impulse response of a ray-based propagation of an impulse emitted at the at least one directive sound source for emitting sound in a defined direction in the 3D model of the room and received at the listening point within a second acoustic frequency range; and

generate the impulse response by merging the directional wave-based impulse response and the geometrical impulse response.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2025
From: COSNEFROY, MATTHIAS
To: TREBLE TECHNOLOGIES
Reel/Frame 072016/0594 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2024
From: PIND, FINNUR; PEDERSEN, JESPER
To: TREBLE TECHNOLOGIES
Reel/Frame 067446/0256 →
Priority Claims (3)
EP 22209959 · Nov 28, 2022 · regional
EP 23195400 · Sep 5, 2023 · regional
EP 23204159 · Oct 17, 2023 · regional
Continuity (1)
Related Publication 20240179487A1 · May 30, 2024
References Cited (99)
US 6826483B1 · Anderson · 2004 [cited by examiner]
US 9383464B2 · Shin · 2016 [cited by examiner]
US 9560467B2 · Gorzel et al. · 2017 [cited by applicant]
US 9711126B2 · Mehra · 2017 [cited by examiner]
US 10440498B1 · Gari et al. · 2019 [cited by applicant]
US 10559295B1 · Abel · 2020 [cited by applicant]
US 10777214B1 · Shi et al. · 2020 [cited by applicant]
US 10897570B1 · Robinson et al. · 2021 [cited by applicant]
US 10986444B2 · Mansour et al. · 2021 [cited by applicant]
US 11830471B1 · Mansour et al. · 2023 [cited by applicant]
US 20110015924A1 · Gunel Hacihabiboglu et al. · 2011 [cited by applicant]
US 20150110310A1 · Minnaar · 2015 [cited by applicant]
US 20200214559A1 · Krueger et al. · 2020 [cited by applicant]
US 20200395028A1 · Kameoka et al. · 2020 [cited by applicant]
US 20210074282A1 · Borgstrom et al. · 2021 [cited by applicant]
US 20210074308A1 · Skordilis et al. · 2021 [cited by applicant]
US 20210136510A1 · Tang et al. · 2021 [cited by applicant]
US 20220051479A1 · Agarwal et al. · 2022 [cited by applicant]
US 20220079499A1 · Doron et al. · 2022 [cited by applicant]
US 20220101126A1 · Bharitkar · 2022 [cited by applicant]
US 20220327316A1 · Grauman et al. · 2022 [cited by applicant]
US 20220405602A1 · Yoo et al. · 2022 [cited by applicant]
US 20230164509A1 · Sporer · 2023 [cited by applicant]
US 20230197043A1 · Martinez Ramirez et al. · 2023 [cited by applicant]
US 20230362572A1 · Jang et al. · 2023 [cited by applicant]
WO 2022167720A1 · 2022 [cited by applicant]
Aretz, Combined wave and ray based room acoustic simulation (Year: 2012). [cited by examiner]
Thomas, Open Sphere Cardioid Microphone Array (Year: 2019). [cited by examiner]
Aretz, Combined wave and ray based room acoustic simuation, 2012. [cited by examiner]
Thomas, Open Sphere Cardioid Microphone Array, 2019. [cited by examiner]
Ahrens, Computation of spherical harmonics based sound source directivity models from sparse measurement data, 2020. [cited by examiner]
Anonymous and Others. Hybrid Model for Acoustic Simulation. May 15, 2021, XP93044320. [cited by applicant]
Funkhouser, T. Survey of Methods for Modeling Sound Propagation in Interactive Virtual Environment Systems. Department of Computer Science of Princeton University, Jan. 1, 2003, XP055746257. [cited by applicant]
Hart et al. Machine-learning of long-range sound propagation through simulated atmospheric turbulencea. The Journal of the Acoustical Society of America, American Institute of Physics, vol. 149, No. 6, Jun. 21, 2021, pp… [cited by applicant]
Yeh et al. Using Machine Learning to Predict Indoor Acoustic Indicators of Multi-Functional Activity Centers. Applied Sciences, vol. 11, No. 12, Jun. 18, 2021, p. 5641, XP93054209. [cited by applicant]
Yeh et al. Wave-ray coupling for interactive sound propagation in large complex scenes. ACM Transactions on Graphics, ACM, NY, US, vol. 32, No. 6, Nov. 1, 2013, pp. 1-11, XP058033914. [cited by applicant]
Atkins, H.L et al, “Quadrature-Free Implementation of Discontinuous Galerkin Method for Hyperbolic Equations”. In: AIAA Journal 36.5 (1998), pp. 775-782. [cited by applicant]
Berland, J. et al, “Low-dissipation and low-dispersion fourth-order Runge-Kutta algorithm”, Computers & Fluids 35.10 (2006), pp. 1459-1463. [cited by applicant]
Bilbao, S. et al, “Local time-domain spherical harmonic spatial encoding for wave-based acoustic simulation”. IEEE Signal Processing Letters 26.4 (2019), pp. 617-621. [cited by applicant]
Cosnefroy, M. “Propagation of impulsive sounds in the atmosphere: numerical simulations and comparison with experiments”. PhD thesis. École Centrale de Lyon, 2019. [cited by applicant]
Dragna, D. et al “A generalized recursive convolution method for time-domain propagation in porous media”. In: The Journal of the Acoustical Society of America 138.2 (2015), pp. 1030-1042. [cited by applicant]
Gabard, G. et al “A full discrete dispersion analysis of time-domain simulations of acoustic liners with flow”. In: Journal of Computational Physics 273 (2014), pp. 310-326. [cited by applicant]
Hesthaven, J.S. et al, Nodal Discontinuous Galerkin Methods—Algorithms, Analysis, and Applications (Springer, New York, 2008), Chap. 3. [cited by applicant]
Hu, F.Q. et al, “Low-dissipation and low-dispersion Runge-Kutta schemes for computational acoustics”, Journal of Computational Physics 124.1 (1996), pp. 177-191. [cited by applicant]
Jameson, A. et al “Solution of the Euler equations for complex configurations”, 6th Computational Fluid Dynamics Conference. American Institute of Aeronautics and Astronautics (AIAA), pp. 1-11, 1983. [cited by applicant]
Kuttruff, H. “Room Acoustics: 6th edition”, Dec. 10, 2019. CRC Press. [cited by applicant]
Pind Jorgensson, F. K, Wave-Based Virtual Acoustics. Technical University of Denmark, (2020), 194 pages. [cited by applicant]
Pind, F. et al. “Time domain room acoustic simulations using the spectral element method”. In: The Journal of the Acoustical Society of America 145.6 (2019), pp. 3299-3310. [cited by applicant]
Pind, F. et al, “Time-domain room acoustic simulations with extended-reacting porous absorbers using the discontinuous Galerkin method”, The Journal of the Acoustical Society of America 148.5 (2020), pp. 2851-2863. [cited by applicant]
Pind, F. et al, “A phenomenological extended-reaction boundary model for time-domain wave-based acoustic simulations under sparse reflection conditions using a wave splitting method”, Applied Acoustics 172 (2021), p. 10… [cited by applicant]
Savioja, L. et al, Overview of geometrical room acoustic modeling techniques. J. Acoust. Soc. Am., 138, 708-730, 2015. [cited by applicant]
Strøm, E. et al, “Massively Parallel Nodal Discontinous Galerkin Finite Element Method Simulator for Room Acoustics”. MA thesis. Technical University of Denmark, 2020. [cited by applicant]
Wang, H. et al, “Time-domain impedance boundary condition modeling with the discontinuous Galerkin method for room acoustics simulations”. In: The Journal of the Acoustical Society of America 147.4 (2020), pp. 2534-2546. [cited by applicant]
Wang, H. et al “An arbitrary high-order discontinuous Galerkin method with local time-stepping for linear acoustic wave propagation”, The Journal of the Acoustical Society of America 149.1 (2021), pp. 569-580. [cited by applicant]
Abadi, M. et al, “TensorFlow: A system for large-scale machine learning”, uploaded May 31, 2016, arXiv:1605.08695v2, published in Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation,… [cited by applicant]
Anonymous, “Hybrid Model for Acoustic Simulation” May 15, 2021, pp. 1-6, XP93044320, obtained from Internet: https://reuk.github.io/wayverb/hybrid.html. [cited by applicant]
Yeh C-Y. et al., “Using Machine Learning to Predict Indoor Acoustic Indicators of Multi-Functional Activity Centers”, Article, Applied Sciences, vol. 11, No. 12, Submitted May 28, 2021; Published Jun. 18, 2021, pp. 1-24… [cited by applicant]
Atkins, H.L et al, “Quadrature-Free Implementation of Discontinuous Galerkin Method for Hyperbolic Equations”, AIAA Journal vol. 36, No. 5, May 1998, pp. 775-782, Downloaded by North Dakota State University. [cited by applicant]
Bank, D. et al., “Autoencoders”, Version 2, Submitted Apr. 3, 2021, pp. 1-22, Obtained from Internet: https://arxiv.org/abs/2003.05991v2. [cited by applicant]
Bansal, M. et al, “First Approach to Combine Particle Model Algorithms with Modal Analysis using FEM”, Conventional Paper 6392, AES Convention 118, May 28-31, 2005, Barcelona, Spain, pp. 1-9,AES. [cited by applicant]
Bilbao S. et al, “Local time-domain spherical harmonic spatial encoding for wave-based acoustic simulation”, IEEE Signal Processing Letters, 26.4, Mar. 1, 2019, pp. 617-621, obtained from Internet: https://www.research.… [cited by applicant]
Cosnefroy, M. “Propagation of impulsive sounds in the atmosphere: numerical simulations and comparison with experiments”, Partly in French, PhD thesisPHD thesis, Ecole Centrale de Lyon, Submitted Dec. 18, 2019, pp. 1-22… [cited by applicant]
Denk, F. et al, “Equalization filter design for achieving acoustic transparency in a semi-open fit hearing device”, Speech Communication; 13th ITG-Symposium, Oct. 10-12, 2018, Oldenburg, Germany, pp. 226-230. [cited by applicant]
Dozat,T., “Incorporating Nesterov Momentum into Adam”, Workshop track poster, ICLR May 2, 2016, pp. 1-4. [cited by applicant]
Dragna, D. et al “A generalized recursive convolution method for time-domain propagation in porous media”, The Journal of the Acoustical Society of America 138.2, published online Aug. 20, 2015, pp. 1030-1042, https://d… [cited by applicant]
Funkhouser, T., “Survey of Methods for Modeling Sound Propagation in Interactive Virtual Environment Systems”, Department of Computer Science of Princeton University, Jan. 1, 2003, pp. 1-53, XP055746257. [cited by applicant]
Gabard, G. et al “A full discrete dispersion analysis of time-domain simulations of acoustic liners with flow”, Manuscript, Journal of Computational Physics 273, Received date Nov. 25, 2013, Accepted date May 2, 2014, p… [cited by applicant]
Hart, C. et al., “Machine-learning of long-range sound propagation through simulated atmospheric turbulence”, article, The Journal of the Acoustical Society of America, American Institute of Physics, vol. 149, No. 6, pu… [cited by applicant]
Hesthaven, J.S. et al, “Nodal Discontinuous Galerkin Methods, Algorithms, Analysis, and Applications”, Texts in Applied Mathematics, Chapter 3, pp. 1-507, Springer, New York, 2008. [cited by applicant]
Hu, F.Q. et al, “Low-dissipation and low-dispersion Runge-Kutta schemes for computational acoustics”, Article No. 0052, Journal of Computational Physics 124, 1996, received Dec. 23, 1994, Revised Jul. 1995, pp. 177-191,… [cited by applicant]
Jameson, A. et al “Solution of the Euler equations for complex configurations”, 6th Computational Fluid Dynamics Conference, paper No. 83-1929, pp. 1-11, 1983, American Institute of Aeronautics and Astronautics (AIAA), … [cited by applicant]
Käser, M. et al, “An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes—I. The two-dimensional isotropic case with external source terms”, Journal compilation, Geophys. J. Int. (… [cited by applicant]
Ketkar, N., “Introduction to Keras”, Deep learning with python: a hands-on introduction, Chapter 7, pp. 97-111, 2017, Kikhil Ketkar. [cited by applicant]
Kuttruff, H., “Room Acoustics: 6th edition”, Dec. 10, 2019, pp. 1-302, CRC Press. [cited by applicant]
Majumder, S. et al, “Few-Shot Audio-Visual Learning of Environment Acoustics”, 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Nov. 22, 2022, p. 1-17, arXiv:2206.04006v2. [cited by applicant]
Melander, A. et al, “Massively parallel nodal discontinous Galerkin finite element method simulator for room acoustics”, Research Paper, The International Journal of High Performance Computing Applications 2023, vol. 0(… [cited by applicant]
Miccini, R. et al., “A hybrid approach to structural modeling of individualized HRTFs”, 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Mar. 27, 2021, pp. 80-85, IEEE. [cited by applicant]
Milo, A. et al., “Treble Auralizer: a real time Web Audio Engine enabling 3DoF auralization of simulated room acoustics designs”, Presented at conference 2023 Immersive and 3D Audio: from Architecture to Automotive (13D… [cited by applicant]
Moreau, S. et al. ,“Study of Higher Order Ambisonic Microphone”, CFA/DAGA'04, Strasbourg, Mar. 24-25, 2004, pp. 215-216. [cited by applicant]
Pind Jörgensson, F. K., “Wave-Based Virtual Acoustics”, Ph.D. Thesis, 2020, pp. 1-195, Technical University of Denmark. [cited by applicant]
Pind, F. et al, “A phenomenological extended-reaction boundary model for time-domain wave-based acoustic simulations under sparse reflection conditions using a wave splitting method”, preprint submitted to Applied Acous… [cited by applicant]
Pind, F. et al. “Time domain room acoustic simulations using the spectral element method”, The Journal of the Acoustical Society of America 145.6, 2019, pp. 3299-3310, Acoustical Society of America. [cited by applicant]
Pind, F. et al, “Time-domain room acoustic simulations with extended-reacting porous absorbers using the discontinuous Galerkin method”, The Journal of the Acoustical Society of America 148.5, Nov. 24, 2020, pp. 2851-28… [cited by applicant]
Ratnarajah, A. et al., “IR-GAN: Room impulse response generator for far-field speech recognition”, Interspeech 2021, 22nd Annual Conference of the International Speech Communication Association, Brno, Czechia, Aug. 30-S… [cited by applicant]
Reed, W. H. et al., “Triangular mesh methods for the neutron transport equation”, Submitted to Proceedings of the American Nuclear Society by Los Alamos Scientific Laboratory, Oct. 31, 1973, pp. 1-23. [cited by applicant]
Richard, A. et al., “Deep Impulse Responses: Estimating and Parameterizing Filters With Deep Networks”, Feb. 7, 2022, pp. 1-5, arXiv:2202.03416v1 [cs.SD], obtained from Internet: https://arxiv.org/abs/2202.03416v1. [cited by applicant]
Rumelhart, D.E. et al, “Learning Internal Representations by Error Propagation”, Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations, Chapter 8, 1987, pp. 318-362, MIT Press. [cited by applicant]
Sakamoto, S. et al, “Calculation of impulse responses and acoustic parameters in a hall by the finite-difference time-domain method”, Acoust. Sci. & Tech. 29, 4, 2008, accepted Feb. 1, 2008, pp. 256-265, The Acoustical … [cited by applicant]
Sanaguano-Moreno, D.A. et al, “A Deep Learning approach for the Generation of Room Impulse Responses”, 2022 Third International Conference of Information Systems and Software Technologies (ICI2ST), IEEE, Nov. 8, 2022, p… [cited by applicant]
Savioja, L. et al, “Overview of geometrical room acoustic modeling techniques”, J. Acoust. Soc. Am., 138, published online Aug. 10, 2015, pp. 708-730. [cited by applicant]
Singh, N. et al., “Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis”, submitted Aug. 13, 2021, pp. 1-22, arXiv:2103.14201v2 [cs.SD], obtained from internet: https://arxiv.org/abs/2103.14201. [cited by applicant]
Strøm, E. et al., “Massively Parallel Nodal Discontinous Galerkin Finite Element Method Simulator for Room Acoustics”, Master thesis, Apr. 2020, pp. 1-133, Technical University of Denmark. [cited by applicant]
Yeh C-Y. et al., “Wave-ray coupling for interactive sound propagation in large complex scenes”, ACM Transactions on Graphics, ACM, NY, US, vol. 32, No. 6, Article 165, Nov. 2013, pp. 1-11, XP058033914. [cited by applicant]
Wang, H. et al., “Time-domain impedance boundary condition modeling with the discontinuous Galerkin method for room acoustics simulations”, The Journal of the Acoustical Society of America 147.4, 2020, pp. 2534-2546, Ac… [cited by applicant]
Wang, H. et al., “An arbitrary high-order discontinuous Galerkin method with local time-stepping for linear acoustic wave propagation”, The Journal of the Acoustical Society of America 149.1, publication date Jan. 25, 2… [cited by applicant]
Xu, Z. et al, “Simulating room transfer functions between transducers mounted on audio devices using a modified image source method”, J. Coust. Soc. Am., Sep. 8, 2023, Submitted Sep. 7, 2023, arXiv:2309.03486 [eess.AS],… [cited by applicant]
Sakamoto, S. et al., “Directional sound source modeling by using spherical harmonic functions for finite-difference time-domain analysis”, Proceedings of Meetings on Acoustics, vol. 19, 2013, ICA 2013 Montreal, Jun. 2-7… [cited by applicant]
Pind, F. et al., “A novel wave-based virtual acoustics and spatial audio framework”, Audio Engineering Society Conference Paper, AVAR Conference, Richmond, VA, Aug. 15-17, 2022, pp. 1-10, AES. [cited by applicant]
Rocchesso, D., “Maximally Diffusive Yet Efficient Feedback Delay Networks for Artificial Reverberation”, IEEE Signal Processing Letters, vol. 4, No. 9, Sep. 1997, pp. 252-255, IEEE. [cited by applicant]