IP Library Granted Patent US 12,294,850
Granted Patent B2
US 12,294,850 · App. 18/663,637 · Granted May 6, 2025

Audio processing apparatus and method, and program

Inventors: Yuki Yamamoto (Tokyo, JP); Toru Chinen (Kanagawa, JP); Minoru Tsuji (Chiba, JP)
Assignee: Sony Group Corporation
H04S7/303G10L19/008H04S3/008H04S5/02H04S2400/01H04S2400/11H04S2400/13H04S2400/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,294,850
App. No.
18/663,637
Granted
May 6, 2025
Kind
B2
Abstract

The present technology relates to an audio processing apparatus and method and a program that make it possible to obtain sound of higher quality. An acquisition unit acquires an audio signal and metadata of an object. A vector calculation unit calculates, based on a horizontal direction angle and a vertical direction angle included in the metadata of the object and indicative of an extent of a sound image, a spread vector indicative of a position in a region indicative of the extent of the sound image. A gain calculation unit calculates, based on the spread vector, a VBAP gain of the audio signal in regard to each speaker by VBAP. The present technology can be applied to an audio processing apparatus.

Claims (17)

1. An audio processing apparatus comprising:

an acquisition unit configured to acquire metadata including position information indicative of a position of an audio object and sound image information configured from a vector of two or more dimensions and representative of an extent of a sound image from the position;

a vector calculation unit configured to calculate, based on a horizontal direction angle and a vertical direction angle of a region representative of the extent of the sound image determined by the sound image information, a spread vector indicative of a position in the region; and

a gain calculation unit configured to calculate, based on the spread vector and using vector base amplitude panning (VBAP), a gain of each of audio signals supplied to two or more sound outputting units positioned in the proximity of the position indicated by the position information, wherein

the gain calculation unit calculates the gain for each spread vector in regard to each of the sound outputting units, calculates an addition value of the gains calculated in regard to the spread vectors for each of the sound outputting units, normalizes the addition value, and calculates a final gain for each of the sound outputting units based on the normalized addition value.

2. An audio processing method comprising the steps of:

acquiring metadata including position information indicative of a position of an audio object and sound image information configured from a vector of two or more dimensions and representative of an extent of a sound image from the position;

calculating, based on a horizontal direction angle and a vertical direction angle of a region representative of the extent of the sound image determined by the sound image information, a spread vector indicative of a position in the region; and

calculating, based on the spread vector and using vector base amplitude panning (VBAP), a gain of each of audio signals supplied to two or more sound outputting units positioned in the proximity of the position indicated by the position information, wherein

the calculating the gain including:

calculating the gain for each spread vector in regard to each of the sound outputting units, calculating an addition value of the gains calculated in regard to the spread vectors for each of the sound outputting units, normalizing the addition value, and calculating a final gain for each of the sound outputting units based on the normalized addition value.

3. A non-transitory computer readable storage medium having computer readable instructions stored thereon that, when executed by a processor, cause a computer to execute a process comprising the steps of:

acquiring metadata including position information indicative of a position of an audio object and sound image information configured from a vector of two or more dimensions and representative of an extent of a sound image from the position;

calculating, based on a horizontal direction angle and a vertical direction angle of a region representative of the extent of the sound image determined by the sound image information, a spread vector indicative of a position in the region; and

calculating, based on the spread vector and using vector base amplitude panning (VBAP), a gain of each of audio signals supplied to two or more sound outputting units positioned in the proximity of the position indicated by the position information, wherein

the calculating the gain including:

calculating the gain for each spread vector in regard to each of the sound outputting units, calculating an addition value of the gains calculated in regard to the spread vectors for each of the sound outputting units, normalizing the addition value, and calculating a final gain for each of the sound outputting units based on the normalized addition value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2024
From: YAMAMOTO, YUKI; CHINEN, TORU; TSUJI, MINORU
To: SONY CORPORATION
Reel/Frame 068389/0451 →
CHANGE OF NAME Recorded Jul 22, 2024
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 068389/0455 →
Priority Claims (2)
JP 2015-126650 · Jun 24, 2015 · national
JP 2015-148683 · Jul 28, 2015 · national
Continuity (5)
Continuation 17993001 · Nov 23, 2022
Continuation 17474669 · Sep 14, 2021
Continuation 16734211 · Jan 3, 2020
Continuation 15737026
Related Publication 20240298137A1 · Sep 5, 2024
References Cited (55)
US 5046097A · Lowe et al. · 1991 [cited by applicant]
US 10567903B2 · Yamamoto et al. · 2020 [cited by applicant]
US 11140505B2 · Yamamoto et al. · 2021 [cited by applicant]
US 11540080B2 · Yamamoto et al. · 2022 [cited by applicant]
US 20100157726A1 · Ando et al. · 2010 [cited by applicant]
US 20120237062A1 · Korn · 2012 [cited by applicant]
US 20140023197A1 · Xiang et al. · 2014 [cited by applicant]
US 20140119581A1 · Tsingos et al. · 2014 [cited by applicant]
US 20160028633A1 · Durand et al. · 2016 [cited by applicant]
US 20160133261A1 · Shi · 2016 [cited by applicant]
US 20160165374A1 · Shi · 2016 [cited by applicant]
US 20160286332A1 · Shi · 2016 [cited by applicant]
US 20160286333A1 · Robinson et al. · 2016 [cited by applicant]
US 20180160250A1 · Yamamoto et al. · 2018 [cited by applicant]
US 20200145777A1 · Yamamoto et al. · 2020 [cited by applicant]
US 20210409892A1 · Yamamoto et al. · 2021 [cited by applicant]
US 20230078121A1 · Yamamoto et al. · 2023 [cited by applicant]
BR 8904422A · 1990 [cited by applicant]
CA 1037877A · 1978 [cited by applicant]
CA 2279117A1 · 2000 [cited by applicant]
CN 1672464A · 2005 [cited by applicant]
CN 1976546A · 2007 [cited by applicant]
CN 101484935A · 2009 [cited by applicant]
CN 103650535A · 2014 [cited by applicant]
CN 104604254A · 2015 [cited by applicant]
EP 2458881A2 · 2012 [cited by applicant]
EP 2458895A2 · 2012 [cited by applicant]
JP 2006128816A · 2006 [cited by applicant]
JP 2008124639A · 2008 [cited by applicant]
JP 2010536299A · 2010 [cited by applicant]
JP 2014090504A · 2014 [cited by applicant]
JP 2014520491A · 2014 [cited by applicant]
JP 2015080119A · 2015 [cited by applicant]
WO WO2014160576A2 · 2014 [cited by applicant]
WO WO2014192602A1 · 2014 [cited by applicant]
WO WO2015012122A1 · 2015 [cited by applicant]
WO WO2015002517A1 · 2015 [cited by applicant]
WO WO2015053109A1 · 2015 [cited by applicant]
International Search Report and English translation thereof mailed Jul. 19, 2016 in connection with International Application No. PCT/JP2016/067195. [cited by applicant]
Written Opinion and English translation thereof mailed Jul. 19, 2016 in connection with International Application No. PCT/JP2016/067195. [cited by applicant]
International Preliminary Report on Patentability and English translation thereof mailed Jan. 4, 2018 in connection with International Application No. PCT/JP2016/067195. [cited by applicant]
Korean Office Action mailed May 3, 2018 in connection with Korean Application No. 10-2017-7035890 and English translation thereof. [cited by applicant]
Extended European Search Report issued Jan. 23, 2019 in connection with European Application No. 16814177.8. [cited by applicant]
Chinese Office Action issued Aug. 22, 2019 in connection with Chinese Application No. 201680034827.1 and English translation thereof. [cited by applicant]
Extended European Search Report dated Mar. 31, 2020 in connection with European Application No. 20155520.8. [cited by applicant]
Japanese Office Action dated Jul. 6, 2020 in connection with Japanese Application No. 2017-525183 and English translation thereof. [cited by applicant]
First Chinese Office Action issued Jan. 30, 2024 in connection with Chinese Application No. 202011538529.0 and English translation thereof. [cited by applicant]
[No Author Listed], Annex—Proposed modifications to the text of ISO/IEC 23008-3, Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 3: 3D audio, ISO/IEC JTC 1/SC 29, Aug.… [cited by applicant]
[No Author Listed], Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 3: 3D audio, ISO/IEC JTC 1/SC 29, Jul. 25, 2014, 433 pages. [cited by applicant]
[No Author Listed], Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 3: 3D audio, Amendment 3: MPEG-H 3D Audio Phase 2, ISO/IEC JTC 1/SC 29/WG 11, 23008-3:2015, Nov. 16… [cited by applicant]
[No Author Listed], Information technology—High efficiency coding and media delivery in heterogeneous environments—Part 3: 3D audio, Amendment 1: MPEG-H, 3D Audio Profiles and Levels, ISO/IEC JTC 1/SC 29/WG 11, 23008-3:… [cited by applicant]
Fueg et al., Metadata Updates to MPEG-H 3D Audio, International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11, Coding of Moving Pictures and Audio, MPEG 2015, M36586, Jun. 2015, Warsaw, Poland, 51 pages. [cited by applicant]
Herre et al., MPEG-H Audio—The New Standard for Universal Spatial/3D Audio Coding, Journal of the Audio Engineering Society, Dec. 2014, vol. 62, No. 12, pp. 821-830. [cited by applicant]
Pulkki V., Uniform Spreading of Amplitude Panned Virtual Sources, Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999, 4 pages. [cited by applicant]
Pulkki V., Virtual Sound Source Positioning Using Vector Base Amplitude Panning, Laboratory of Acoustics and Audio Signal Processing, vol. 45, No. 6, Jun. 1997, pp. 456-466. [cited by applicant]