IP Library › Granted Patent US 11,341,952
Granted Patent B2
US 11,341,952 · App. 16/985,734 · Granted May 24, 2022

System and method for generating audio featuring spatial representations of sound sources

Inventors: Ron Ziv (Kfar-Saba, IL); Tomer Goshen (Hod Hasharon, IL); Emil Winebrand (Petah Tikva, IL); Yadin Aharoni (Tel Aviv, IL)
Assignee: Insoundz, Ltd.
G10L13/02G10L19/02H04R3/005H04R5/04H04S7/303
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,341,952
App. No.
16/985,734
Granted
May 24, 2022
Kind
B2
Abstract

Systems and methods for spatially emulating a sound source. An apparatus includes a microphone array including microphones; and a sound profiler communicatively connected to the microphone array, the sound profiler including a processing circuitry and a memory which contains instructions that, when executed by the processing circuitry, configure the apparatus to: generate synthesized audio based on sound beam metadata, a sound profile, and target listener location data, wherein the sound beam metadata includes timed sound beams defining a directional dependence of a spatial sound wave, wherein the sound profile includes timed sound coefficients determined based on audio signals captured in a space wherein the target listener location data includes a position and an orientation, wherein the synthesized audio emulates sound that would be heard by a listener at the position and orientation of the target listener location data; and providing the synthesized audio for projection.

Claims (35)

1. A method for spatially emulating a sound source, comprising:

transforming a plurality of timed audio samples by applying a Fast Fourier Transform (FFT) to the plurality of timed audio samples, wherein the plurality of timed audio samples includes a plurality of audio signals captured in a space at respective times;

determining a plurality of relative transfer functions based on a plurality of spatial base functions, wherein the plurality of relative transfer functions is a plurality of second transfer functions, wherein the plurality of second transfer functions is determined based on ratios between first transfer functions of a plurality of first transfer functions, wherein the plurality of first transfer functions is determined based on the plurality of spatial base functions;

generating a plurality of beamforms based on the transformed plurality of audio samples and the plurality of relative transfer functions; and

determining a plurality of timed sound coefficients by applying an inverse FFT to the plurality of beamforms, wherein the plurality of timed sound coefficients produce audio emulating sound that would be heard by a target listener in the space when utilized to generate audio based on a target position and a target orientation of the target listener.

2. The method of claim 1 , wherein generating the plurality of beamforms further comprises:

applying a plurality of spatial base functions to the plurality of timed audio samples.

3. The method of claim 2 , wherein the plurality of spatial base functions includes at least one spherical harmonic function.

4. The method of claim 1 , wherein the plurality of beamforms is generated using any of: minimum variance distortion-less response, generalized side-lobe canceler beam forming, and delay and sum beam forming.

5. The method of claim 1 , further comprising:

transmitting the plurality of timed sound coefficients for use in generating audio.

6. The method of claim 5 , wherein transmitting the plurality of timed sound coefficients further comprises:

storing the plurality of timed sound coefficients in an intermediate storage.

7. The method of claim 5 , wherein the plurality of audio signals is captured by at least one microphone array deployed in the space.

8. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:

transforming a plurality of timed audio samples by applying a Fast Fourier Transform (FFT) to the plurality of timed audio samples, wherein the plurality of timed audio samples includes a plurality of audio signals captured in a space at respective times;

determining a plurality of relative transfer functions based on a plurality of spatial base functions, wherein the plurality of relative transfer functions is a plurality of second transfer functions, wherein the plurality of second transfer functions is determined based on ratios between first transfer functions of a plurality of first transfer functions, wherein the plurality of first transfer functions is determined based on the plurality of spatial base functions;

generating a plurality of beamforms based on the transformed plurality of audio samples and the plurality of relative transfer functions; and

determining a plurality of timed sound coefficients by applying an inverse FFT to the plurality of beamforms, wherein the plurality of timed sound coefficients produce audio emulating sound that would be heard by a target listener in the space when utilized to generate audio based on a target position and a target orientation of the target listener.

9. A system for spatially emulating a sound source, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:

transform a plurality of timed audio samples by applying a Fast Fourier Transform (FFT) to the plurality of timed audio samples, wherein the plurality of timed audio samples includes a plurality of audio signals captured in a space at respective times;

determine a plurality of relative transfer functions based on a plurality of spatial base functions, wherein the plurality of relative transfer functions is a plurality of second transfer functions, wherein the plurality of second transfer functions is determined based on ratios between first transfer functions of a plurality of first transfer functions, wherein the plurality of first transfer functions is determined based on the plurality of spatial base functions;

generate a plurality of beamforms based on the transformed plurality of audio samples and the plurality of relative transfer functions; and

determine a plurality of timed sound coefficients by applying an inverse FFT to the plurality of beamforms, wherein the plurality of timed sound coefficients produce audio emulating sound that would be heard by a target listener in the space when utilized to generate audio based on a target position and a target orientation of the target listener.

10. The system of claim 9 , the system is further configured to:

apply a plurality of spatial base functions to the plurality of timed audio samples.

11. The system of claim 10 , wherein the plurality of spatial base functions includes at least one spherical harmonic function.

12. The system of claim 9 , wherein the plurality of beamforms is generated using any of: minimum variance distortion-less response, generalized side-lobe canceler beam forming, and delay and sum beam forming.

13. The system of claim 9 , the system is further configured to:

transmit the plurality of timed sound coefficients for use in generating audio.

14. The system of claim 13 , the system is further configured to:

store the plurality of timed sound coefficients in an intermediate storage.

15. The system of claim 13 , wherein the plurality of audio signals is captured by at least one microphone array deployed in the space.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 17, 2020
From: ZIV, RON; GOSHEN, TOMER; WINEBRAND, EMIL; AHARONI, YADIN
To: INSOUNDZ LTD.
Reel/Frame 053513/0151 →
Continuity (2)
Provisional Application 62883250 · Aug 6, 2019
Related Publication 20210043185A1 · Feb 11, 2021