IP Library Granted Patent US 9,462,406
Granted Patent B2
US 9,462,406 · App. 14/334,185 · Granted Oct 4, 2016

Method and apparatus for facilitating spatial audio capture with multiple devices

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,462,406
App. No.
14/334,185
Granted
Oct 4, 2016
Kind
B2
Abstract

A method, apparatus and computer program product are provided in order to facilitate spatial audio capture by utilizing multiple audio recording devices. In the context of a method, spatial characteristics of sound recorded by a first device are determined. The method also mapping, with a processor, sound recorded by a second device to a location and orientation of the first device. As such, the spatial characteristics of the sound recorded by the second device are modified to match the spatial characteristics of the sound recorded by the first device.

Claims (26)

1. A method comprising:

determining spatial characteristics of sound recorded by a first device; and

based upon at least one of a relative location and orientation of a second device to the first device, mapping, with a processor, sound recorded by the second device to at least one of a location and orientation of the first device so that the spatial characteristics of the sound recorded by the second device are modified based upon the spatial characteristics of the sound recorded by the first device.

2. A method according to claim 1 wherein the first device is also configured to capture video, and wherein the method further comprises combining the video captured by the first device and the sound recorded by the second device following modification of the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

3. A method according to claim 1 wherein the first device is configured to provide one or more audio channels and an auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device, and wherein mapping the sound recorded by the second device to at least one of the location and orientation of the first device comprises replacing the sound recorded by the first device with the sound recorded by the second device in combination with the auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device.

4. A method according to claim 1 wherein determining the spatial characteristics of the sound recorded by the first device comprises determining the relative location and the relative orientation between the first and second devices, and wherein mapping the sound recorded by the second device to the location and orientation of the first device comprises modifying the spatial characteristics of the sound recorded by the second device based upon the relative location and the relative orientation between the first and second devices.

5. A method according to claim 1 wherein mapping the sound recorded by the second device to at least one of the location and orientation of the first device comprises rotating the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

6. A method according to claim 1 further comprising determining that a sound quality of the sound recorded by the first device fails to satisfy a predefined threshold, wherein mapping the sound recorded by the second device to at least one of the location and orientation of the first device is performed in response to determining that the sound quality of the sound recorded by the first device fails to satisfy the predefined threshold.

7. A method according to claim 6 wherein determining that the sound quality of the sound recorded by the first device fails to satisfy the predefined threshold comprises determining that the sound recorded by the first device includes noise, and wherein mapping the sound recorded by the second device to at least one of the location and orientation of the first device is performed in response to determining that the sound recorded by the first device includes noise.

8. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least:

determine spatial characteristics of sound recorded by a first device; and

based upon at least one of a relative location and orientation of a second device to the first device, map sound recorded by the second device to at least one of a location and orientation of the first device so that the spatial characteristics of the sound recorded by the second device are modified based upon the spatial characteristics of the sound recorded by the first device.

9. An apparatus according to claim 8 wherein the first device is also configured to capture video, and wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to combine the video captured by the first device and the sound recorded by the second device following modification of the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

10. An apparatus according to claim 8 wherein the first device is configured to provide one or more audio channels and an auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to map the sound recorded by the second device to at least one of the location and orientation of the first device by replacing the sound recorded by the first device with the sound recorded by the second device in combination with the auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device.

11. An apparatus according to claim 8 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to determine the spatial characteristics of the sound recorded by the first device by determining the relative location and the relative orientation between the first and second devices, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to map the sound recorded by the second device to the location and orientation of the first device by modifying the spatial characteristics of the sound recorded by the second device based upon the relative location and the relative orientation between the first and second devices.

12. An apparatus according to claim 8 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to map the sound recorded by the second device to at least one of the location and orientation of the first device by rotating the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

13. An apparatus according to claim 8 wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to determine that a sound quality of the sound recorded by the first device fails to satisfy a predefined threshold, wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to map the sound recorded by the second device to the at least one of the location and orientation of the first device in response to determining that the sound quality of the sound recorded by the first device fails to satisfy the predefined threshold.

14. An apparatus according to claim 13 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to determine that the sound quality of the sound recorded by the first device fails to satisfy the predefined threshold by determining that the sound recorded by the first device includes noise, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to map the sound recorded by the second device to at least one of the location and orientation of the first device in response to determining that the sound recorded by the first device includes noise.

15. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions configured to:

determine spatial characteristics of sound recorded by a first device; and

based upon at least one of a relative location and orientation of a second device to the first device, map sound recorded by the second device to at least one of a location and orientation of the first device so that the spatial characteristics of the sound recorded by the second device are modified based upon the spatial characteristics of the sound recorded by the first device.

16. A computer program product according to claim 15 wherein the first device is also configured to capture video, and wherein the computer-executable program code portions further comprise program code instructions configured to combine the video captured by the first device and the sound recorded by the second device following modification of the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

17. A computer program product according to claim 15 wherein the first device is configured to provide one or more audio channels and an auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device, and wherein the program code instructions configured to map the sound recorded by the second device to at least one of the location and orientation of the first device comprise program code instructions configured to replace the sound recorded by the first device with the sound recorded by the second device in combination with the auxiliary data stream that includes the spatial characteristics of the sound recorded by the first device.

18. A computer program product according to claim 15 wherein the program code instructions configured to determine the spatial characteristics of the sound recorded by the first device comprise program code instructions configured to determine the relative location and relative orientation between the first and second devices, and wherein the program code instructions configured to map the sound recorded by the second device to the location and orientation of the first device comprise program code instructions configured to modify the spatial characteristics of the sound recorded by the second device based upon the relative location and the relative orientation between the first and second devices.

19. A computer program product according to claim 15 wherein the program code instructions configured to map the sound recorded by the second device to at least one of the location and orientation of the first device comprise program code instructions configured to rotate the spatial characteristics of the sound recorded by the second device based upon the spatial characteristics of the sound recorded by the first device.

20. A computer program product according to claim 15 wherein the computer-executable program code portions further comprise program code instructions configured to determine that a sound quality of the sound recorded by the first device fails to satisfy a predefined threshold, wherein the program code instructions configured to map the sound recorded by the second device to at least one of the location and orientation of the first device is performed in response to determining that the sound quality of the sound recorded by the first device fails to satisfy the predefined threshold.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 034781/0200 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE ADDRESS INSIDE ASSIGNMENT PREVIOUSLY RECORDED AT REEL: 033336 FRAME: 0243. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Nov 7, 2014
From: HIIPAKKA, JARMO ANTERO; TAMMI, MIKKO TAPIO; VESA, SAMPO; UUSITALO, JUSSI SEVERI; KAARESOJA, TOPI
To: NOKIA CORPORATION
Reel/Frame 034192/0834 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2014
From: HIIPAKKA, JARMO ANTERO; TAMMI, MIKKO TAPIO; VESA, SAMPO; UUSITALO, JUSSI SEVERI; KAARESOJA, TOPI
To: NOKIA CORPORATION
Reel/Frame 033336/0243 →