IP Library Granted Patent US 12,289,483
Granted Patent B2
US 12,289,483 · App. 18/407,888 · Granted Apr 29, 2025

Encoding device and method, reproduction device and method, and program

Inventors: Minoru Tsuji (Chiba, JP); Toru Chinen (Kanagawa, JP); Runyu Shi (Kanagawa, JP); Masayuki Nishiguchi (Kanagawa, JP); Yuki Yamamoto (Tokyo, JP)
Assignee: Sony Group Corporation
H04N21/234363H04N19/156H04N19/157H04N19/70H04N19/80H04N21/23614H04N21/435H04N21/45455H04N21/4728H04N21/84H04N19/167H04N21/236
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,289,483
App. No.
18/407,888
Granted
Apr 29, 2025
Kind
B2
Abstract

The present technology relates to an encoding device, an encoding method, a reproduction device, a reproduction method, and a program enabling each reproduction equipment to reproduce an appropriate content in a simplified manner. A content data decoding unit decodes encoded metadata and outputs zoom area information, which is included in metadata acquired as a result thereof, designating an area to be zoomed. A zoom area selecting unit selects one or a plurality of pieces of zoom area information from among the zoom area information. A video segmenting unit segments a zoom area represented by the selected zoom area information in a video based on video data and outputs zoom video data acquired as a result thereof. An audio converting unit performs an audio converting process according to the selected zoom area information for audio data and outputs zoom audio data acquired as a result thereof. The present technology can be applied to a reproduction device.

Claims (38)

1. A reproduction device comprising:

processing circuitry configured to:

receive a bit stream including encoded audio data and a plurality of pieces of zoom area information;

decode the encoded audio data;

select one or more pieces of zoom area information from among the plurality of pieces of zoom area information on the basis of information relating to the reproduction device; and

perform an audio converting process for the decoded audio data on the basis of the selected zoom area information,

wherein the audio data is an object-based audio, and

wherein the audio converting process includes moving a position of a sound source object.

2. The reproduction device according to claim 1 , wherein the received bit stream includes encoded video data and wherein the processing circuitry is further configured to:

decode the encoded video data; and

perform a trimming process for the decoded video data on the basis of the selected zoom area information.

3. The reproduction device according to claim 2 , wherein, in the plurality of pieces of zoom area information, the zoom area information designates an area for each specific video object is included.

4. The reproduction device according to claim 1 , wherein the information relating to the reproduction device indicates a type of the reproduction device.

5. The reproduction device according to claim 4 , wherein, in the plurality of pieces of zoom area information, the zoom area information designates an area for each type of reproduction target equipment is included.

6. The reproduction device according to claim 4 , wherein, in the plurality of pieces of zoom area information, the zoom area information designates an area for each rotation direction of reproduction target equipment is included.

7. The reproduction device according to claim 1 , wherein the processing circuitry is configured to select the zoom area information in accordance with a user's operation input.

8. The reproduction device according to claim 1 , wherein the processing circuitry is configured to select the zoom area information by using information representing a rotation direction of the reproduction device as the information relating to the reproduction device.

9. The reproduction device according to claim 1 , wherein the audio converting process includes changing a distance from the sound source object to a watching point.

10. A reproduction method executed by a reproduction device including processing circuitry, the reproduction method comprising:

receiving a bit stream including encoded audio data and a plurality of pieces of zoom area information;

decoding the encoded audio data;

selecting one or more pieces of zoom area information from among the plurality of pieces of zoom area information on the basis of information relating to the reproduction device; and

performing an audio converting process for the decoded audio data on the basis of the selected zoom area information,

wherein the audio data is an object-based audio, and

wherein the audio converting process includes moving a position of a sound source object.

11. The reproduction method according to claim 10 , wherein the received bit stream includes encoded video data, further comprising:

decoding the encoded video data; and

performing a trimming process for the decoded video data on the basis of the selected zoom area information.

12. A non-transitory computer readable medium containing instructions that, when executed by a reproduction device including processing circuitry, perform a reproduction method comprising:

receiving a bit stream including encoded audio data and a plurality of pieces of zoom area information;

decoding the encoded audio data;

selecting one or more pieces of zoom area information from among the plurality of pieces of zoom area information on the basis of information relating to the reproduction device; and

performing an audio converting process for the decoded audio data on the basis of the selected zoom area information,

wherein the audio data is an object-based audio, and

wherein the audio converting process includes moving a position of a sound source object.

13. The non-transitory computer readable medium according to claim 12 , wherein the received bit stream includes encoded video data and wherein the reproduction method further comprises:

decoding the encoded video data; and

performing a trimming process for the decoded video data on the basis of the selected zoom area information.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2024
From: TSUJI, MINORU; CHINEN, TORU; SHI, RUNYU; NISHIGUCHI, MASAYUKI; YAMAMOTO, YUKI
To: SONY CORPORATION
Reel/Frame 066889/0428 →
CHANGE OF NAME Recorded Mar 25, 2024
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 066892/0666 →
Priority Claims (1)
JP 2014-208594 · Oct 10, 2014 · national
Continuity (4)
Continuation 17729251 · Apr 26, 2022
Continuation 16826675 · Mar 23, 2020
Continuation 15516537
Related Publication 20240146981A1 · May 2, 2024
References Cited (138)
US 6714253B2 · Kim et al. · 2004 [cited by applicant]
US 7420598B1 · Ito · 2008 [cited by applicant]
US 7646437B1 · Dumitras et al. · 2010 [cited by applicant]
US 8358363B2 · Arai · 2013 [cited by applicant]
US 8693846B2 · Prestenback et al. · 2014 [cited by applicant]
US 8823667B1 · Hill · 2014 [cited by applicant]
US 8826145B1 · Kirkpatrick et al. · 2014 [cited by applicant]
US 9071830B2 · Okada et al. · 2015 [cited by applicant]
US 9078014B2 · Hendricks et al. · 2015 [cited by applicant]
US 9131098B2 · Jang et al. · 2015 [cited by applicant]
US 9165203B2 · McCarthy · 2015 [cited by applicant]
US 9185268B2 · Doser et al. · 2015 [cited by applicant]
US 9235917B2 · Puri et al. · 2016 [cited by applicant]
US 9329827B2 · Lavine et al. · 2016 [cited by applicant]
US 9491319B2 · Yokoyama et al. · 2016 [cited by applicant]
US 9508172B1 · Hume et al. · 2016 [cited by applicant]
US 9607658B2 · Atkins et al. · 2017 [cited by applicant]
US 9626084B2 · Waggoner et al. · 2017 [cited by applicant]
US 9654751B2 · Doser et al. · 2017 [cited by applicant]
US 9684716B2 · McConville et al. · 2017 [cited by applicant]
US 9870801B2 · Puri et al. · 2018 [cited by applicant]
US 9894422B2 · Hwang et al. · 2018 [cited by applicant]
US 9922007B1 · Jain et al. · 2018 [cited by applicant]
US 10057591B2 · Yuan et al. · 2018 [cited by applicant]
US 10192563B2 · Fueg et al. · 2019 [cited by applicant]
US 10200721B2 · Denoual et al. · 2019 [cited by applicant]
US 10210907B2 · Puri et al. · 2019 [cited by applicant]
US 10397666B2 · Thomas et al. · 2019 [cited by applicant]
US 10582221B2 · Denoual et al. · 2020 [cited by applicant]
US 10587977B2 · Neukam et al. · 2020 [cited by applicant]
US 10595062B2 · Maze et al. · 2020 [cited by applicant]
US 10631025B2 · Tsuji et al. · 2020 [cited by applicant]
US 10742999B2 · Wang et al. · 2020 [cited by applicant]
US 10805614B2 · D'Acunto et al. · 2020 [cited by applicant]
US 10841566B2 · He et al. · 2020 [cited by applicant]
US 11006181B2 · Murtaza et al. · 2021 [cited by applicant]
US 11330310B2 · Tsuji et al. · 2022 [cited by applicant]
US 20020157112A1 · Kuhn · 2002 [cited by applicant]
US 20040117735A1 · Breen · 2004 [cited by applicant]
US 20040212731A1 · Sie et al. · 2004 [cited by applicant]
US 20050093981A1 · Nonaka · 2005 [cited by applicant]
US 20050195205A1 · Abrams · 2005 [cited by applicant]
US 20060061602A1 · Schmouker et al. · 2006 [cited by applicant]
US 20060224448A1 · Herf · 2006 [cited by applicant]
US 20070269187A1 · Morimoto et al. · 2007 [cited by applicant]
US 20090086042A1 · Vartiainen et al. · 2009 [cited by applicant]
US 20090089448A1 · Sze et al. · 2009 [cited by applicant]
US 20090115901A1 · Winter et al. · 2009 [cited by applicant]
US 20090251594A1 · Hua et al. · 2009 [cited by applicant]
US 20100110108A1 · Alexandersson et al. · 2010 [cited by applicant]
US 20100158099A1 · Kalva et al. · 2010 [cited by applicant]
US 20110007131A1 · Okada et al. · 2011 [cited by applicant]
US 20110099494A1 · Yan et al. · 2011 [cited by applicant]
US 20110289410A1 · Paczkowski et al. · 2011 [cited by applicant]
US 20110299832A1 · Butcher · 2011 [cited by applicant]
US 20110311207A1 · Urabe et al. · 2011 [cited by applicant]
US 20120038675A1 · Johnson et al. · 2012 [cited by applicant]
US 20120079578A1 · Dachiraju et al. · 2012 [cited by applicant]
US 20120092348A1 · Mccutchen · 2012 [cited by examiner]
US 20120191876A1 · Johnson et al. · 2012 [cited by applicant]
US 20120210230A1 · Matsuda et al. · 2012 [cited by applicant]
US 20120230512A1 · Ojanpera · 2012 [cited by applicant]
US 20130007818A1 · Kalidindi et al. · 2013 [cited by applicant]
US 20130097634A1 · Jin et al. · 2013 [cited by applicant]
US 20130326343A1 · Phillips et al. · 2013 [cited by applicant]
US 20140009644A1 · Sako · 2014 [cited by examiner]
US 20140086338A1 · Lu et al. · 2014 [cited by applicant]
US 20140185690A1 · Wang et al. · 2014 [cited by applicant]
US 20140196095A1 · Kang · 2014 [cited by applicant]
US 20140219636A1 · Shah · 2014 [cited by applicant]
US 20140280698A1 · Lahti et al. · 2014 [cited by applicant]
US 20150074232A1 · Phillips et al. · 2015 [cited by applicant]
US 20150095804A1 · Grossman et al. · 2015 [cited by applicant]
US 20150131969A1 · Taraki et al. · 2015 [cited by applicant]
US 20150201198A1 · Marlatt et al. · 2015 [cited by applicant]
US 20150201225A1 · Yin · 2015 [cited by applicant]
US 20150205457A1 · Woo et al. · 2015 [cited by applicant]
US 20150208070A1 · Verzijp et al. · 2015 [cited by applicant]
US 20150256861A1 · Oyman · 2015 [cited by applicant]
US 20150296232A1 · Hwang et al. · 2015 [cited by applicant]
US 20150304693A1 · Hwang et al. · 2015 [cited by applicant]
US 20150309669A1 · Wheeler et al. · 2015 [cited by applicant]
US 20150309703A1 · Robertson et al. · 2015 [cited by applicant]
US 20150373341A1 · Davies · 2015 [cited by applicant]
US 20160057509A1 · Yamagishi et al. · 2016 [cited by applicant]
US 20160088319A1 · Zucchetta · 2016 [cited by applicant]
US 20160165309A1 · Van Brandenburg et al. · 2016 [cited by applicant]
US 20160227228A1 · Pomeroy et al. · 2016 [cited by applicant]
US 20160227285A1 · Voss · 2016 [cited by applicant]
US 20160337706A1 · Hwang et al. · 2016 [cited by applicant]
US 20170011751A1 · Fueg et al. · 2017 [cited by applicant]
US 20170127127A1 · Zavesky et al. · 2017 [cited by applicant]
US 20170230546A1 · Lebrun et al. · 2017 [cited by applicant]
US 20170257679A1 · Lopez Zapico · 2017 [cited by applicant]
US 20170353704A1 · Su et al. · 2017 [cited by applicant]
US 20180007407A1 · Maze et al. · 2018 [cited by applicant]
US 20180220204A1 · Yamagishi et al. · 2018 [cited by applicant]
US 20180241925A1 · Suzuki et al. · 2018 [cited by applicant]
US 20180242030A1 · Tsuji et al. · 2018 [cited by applicant]
US 20190110081A1 · Denoual et al. · 2019 [cited by applicant]
US 20190139562A1 · Fueg et al. · 2019 [cited by applicant]
US 20190147914A1 · Puri et al. · 2019 [cited by applicant]
US 20190158935A1 · Denoual et al. · 2019 [cited by applicant]
US 20190320189A1 · Cooper et al. · 2019 [cited by applicant]
US 20200126582A1 · Yamamoto et al. · 2020 [cited by applicant]
US 20200137506A1 · Lee et al. · 2020 [cited by applicant]
US 20200221146A1 · Tsuji et al. · 2020 [cited by applicant]
US 20200228844A1 · Denoual et al. · 2020 [cited by applicant]
US 20200245032A1 · Murtaza et al. · 2020 [cited by applicant]
US 20220256216A1 · Tsuji et al. · 2022 [cited by applicant]
CN 1402539A · 2003 [cited by applicant]
CN 102244807A · 2011 [cited by applicant]
CN 102685597A · 2012 [cited by applicant]
CN 102855464A · 2013 [cited by applicant]
EP 1814360A2 · 2007 [cited by applicant]
JP 2002171529A · 2002 [cited by applicant]
JP 2008199370A · 2008 [cited by applicant]
JP 2010232814A · 2010 [cited by applicant]
JP 2012004835A · 2012 [cited by applicant]
JP 2012060575A · 2012 [cited by applicant]
WO WO2011064438A1 · 2011 [cited by applicant]
WO WO2014036085A1 · 2014 [cited by applicant]
International Search Report and English translation thereof mailed Nov. 10, 2015 in connection with International Application No. PCT/JP2015/077243. [cited by applicant]
Written Opinion and English translation thereof mailed Nov. 10, 2015 in connection with International Application No. PCT/JP2015/077243. [cited by applicant]
International Preliminary Report on Patentability and English translation thereof mailed Apr. 20, 2017 in connection with International Application No. PCT/JP/2015/077243. [cited by applicant]
Extended European Search Report issued Mar. 23, 2018 in connection with European Application No. 15849654.7. [cited by applicant]
Communication pursuant to Article 94(3) issued Feb. 4, 2019 in connection with European Application No. 15849654.7. [cited by applicant]
Japanese Office Action mailed Apr. 2, 2019 in connection with Japanese Application No. 2016-553047 and English translation thereof. [cited by applicant]
Chinese Office Action issued Jun. 28, 2019 in connection with Chinese Application No. 201580053817.8 and English translation thereof. [cited by applicant]
Japanese Office Action mailed Oct. 13, 2020 in connection with Japanese Application No. 2019-142166 and English translation thereof. [cited by applicant]
Extended European Search Report issued Apr. 30, 2021 in connection with European Application No. 20215659.2. [cited by applicant]
Communication pursuant to Article 94(3) EPC dated Jul. 29, 2022 in connection with European Application No. 20215659.2. [cited by applicant]
[No Author Listed], Sony Global—Digital Imaging—FDR-AX100, Sony Corporation, 2014, 7 Pages. http://www.sony.net/Products/di/en-us/products/j4it/index.html accessed [Oct. 7, 2014 14:33:59]. [cited by applicant]
Oldfield et al., Demo paper: Audio object extraction for live sports broadcast. 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) Jul. 15, 2013: 1-2. [cited by applicant]
Shi et al., Screen-Related Object Remapping for Zooming, Coding of Moving Pictures and Audio, ISO/IEC JTC1/SC29/WG11, M34253, Jul. 2014, 12 pages. [cited by applicant]
U.S. Appl. No. 15/516,537, filed Apr. 3, 2017, Tsuji et al. [cited by applicant]
U.S. Appl. No. 16/826,675, filed Mar. 23, 2020, Tsuji et al. [cited by applicant]
U.S. Appl. No. 17/729,251, filed Apr. 26, 2022, Tsuji et al. [cited by applicant]