IP Library › Granted Patent US 12,231,774
Granted Patent B2
US 12,231,774 · App. 18/672,272 · Granted Feb 18, 2025

Method and apparatus for reconstructing 360-degree image according to projection format

Inventor: Ki Baek Kim (Daejeon, KR)
Assignee: B1 INSTITUTE OF IMAGE TECHNOLOGY, INC.
H04N23/698G06T3/40H04N11/28H04N19/103H04N19/11H04N19/119H04N19/124H04N19/129H04N19/13H04N19/134H04N19/159H04N19/174H04N19/176H04N19/186H04N19/30H04N19/31H04N19/33H04N19/44H04N19/503H04N19/51H04N19/625H04N19/45
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,231,774
App. No.
18/672,272
Granted
Feb 18, 2025
Kind
B2
Abstract

Disclosed are methods and apparatuses for image data encoding/decoding. A method for decoding a 360-degree image includes the steps of: receiving a bitstream obtained by encoding a 360-degree image; generating a prediction image by making reference to syntax information obtained from the received bitstream; adding the generated prediction image to a residual image obtained by dequantizing and inverse-transforming the bitstream, so as to obtain a decoded image; and reconstructing the decoded image into a 360-degree image according to a projection format. Therefore, the performance of image data compression can be improved.

Claims (36)

1. A method for decoding a 360-degree image, the method comprising:

receiving a bitstream in which the 360-degree image is encoded, the bitstream including data of an extended 2-dimensional image, the extended 2-dimensional image including a 2-dimensional image and a predetermined extension region, and the 2-dimensional image being from an image with a 3-dimensional projection structure and including one or more faces;

generating a prediction image by referring to syntax information obtained from the received bitstream;

obtaining a decoded image by adding the generated prediction image to a residual image, the residual image being obtained by inverse-quantizing and inverse transforming quantized transform coefficients from the bitstream; and

reconstructing the decoded image into the 360-degree image according to a projection format,

wherein the projection format is selectively determined based on identification information, among a plurality of pre-defined projection formats including an ERP format in which the 360-degree image is projected in a two-dimensional plane or a CMP format in which the 360-degree image is projected in a cube,

wherein a size of the extension region is variably determined based on at least one of first information indicating a width of a left side of the extension region, second information indicating a width of a right side of the extension region, or third information indicating a height of the extension region, independently of a size of the 2-dimensional image,

wherein the extension region is not included in the image with the 3-dimensional projection structure, and

wherein at least one of the identification information, the first information, the second information or the third information is obtained from the bitstream.

2. The method of claim 1 , wherein reconstructing the decoded image comprises:

obtaining, from the bitstream, arrangement information according to a regional packing, the arrangement information specifying one of a plurality of conversion types pre-defined in a decoding apparatus; and

rearranging at least one of a plurality of faces in the decoded image according to the arrangement information.

3. The method of claim 2 , wherein the pre-defined conversion types include at least one of 90-degree rotation, 180-degree rotation, 270-degree rotation, horizontal flipping, or vertical flipping.

4. The method of claim 1 , wherein the extension region is adjacent to a boundary of at least one of the faces.

5. The method of claim 4 , wherein the boundary of the face is discontinuous in the 2-dimensional image and continuous in the image with the 3-dimensional projection structure.

6. A method for encoding a 360-degree image, the method comprising:

generating a bitstream in which the 360-degree image is encoded, the bitstream including data of an extended 2-dimensional image, the extended 2-dimensional image including a 2-dimensional image and a predetermined extension region, and the 2-dimensional image being from an image with a 3-dimensional projection structure and including one or more faces;

wherein the bitstream comprises syntax information, the syntax information being referred to for generating a prediction image,

the bitstream comprises quantized transform coefficients, the quantized transform coefficients being inverse-quantized and inverse transformed for obtaining a residual image,

the prediction image is added to the residual image for obtaining a decoded image, and

the decoded image is reconstructed into the 360-degree image according to a projection format,

wherein the projection format is selectively determined based on identification information, among a plurality of pre-defined projection formats including an ERP format in which the 360-degree image is projected in a two-dimensional plane or a CMP format in which the 360-degree image is projected in a cube,

wherein a size of the extension region is variably determined based on at least one of first information indicating a width of a left side of the extension region, second information indicating a width of a right side of the extension region, or third information indicating a height of the extension region, independently of a size of the 2-dimensional image,

wherein the extension region is not included in the image with the 3-dimensional projection structure, and

wherein at least one of the identification information, the first information, of the second information or the third information is encoded into the bitstream.

7. A method of transmitting a bitstream, the method comprising:

transmitting the bitstream to an image decoding apparatus;

wherein the bitstream is generated by encoding a 360-degree image, the bitstream including data of an extended 2-dimensional image, the extended 2-dimensional image including a 2-dimensional image and a predetermined extension region, and the 2-dimensional image being from an image with a 3-dimensional projection structure and including one or more faces;

wherein the bitstream comprises syntax information, the syntax information being referred to for generating a prediction image,

the bitstream comprises quantized transform coefficients, the quantized transform coefficients being inverse-quantized and inverse transformed for obtaining a residual image,

the prediction image is added to the residual image for obtaining a decoded image, and

the decoded image is reconstructed into the 360-degree image according to a projection format,

wherein the projection format is selectively determined based on identification information, among a plurality of pre-defined projection formats including an ERP format in which the 360-degree image is projected in a two-dimensional plane or a CMP format in which the 360-degree image is projected in a cube,

wherein a size of the extension region is variably determined based on at least one of first information indicating a width of a left side of the extension region, second information indicating a width of a right side of the extension region, or third information indicating a height of the extension region, independently of a size of the 2-dimensional image,

wherein the extension region is not included in the image with the 3-dimensional projection structure, and

wherein at least one of the identification information, the first information, the second information or the third information is encoded into the bitstream.

Priority Claims (3)
KR 10-2016-0127878 · Oct 4, 2016 · national
KR 10-2016-0129382 · Oct 6, 2016 · national
KR 10-2017-0090612 · Jul 17, 2017 · national
Continuity (5)
Continuation 18466442 · Sep 13, 2023
Continuation 17487277 · Sep 28, 2021
Continuation 16372237 · Apr 1, 2019
Continuation PCTKR2017011143 · Oct 10, 2017
Related Publication 20240314442A1 · Sep 19, 2024
References Cited (111)
US 6141034A · McCutchen · 2000 [cited by applicant]
US 6157396A · Margulis et al. · 2000 [cited by applicant]
US 7126630B1 · Lee et al. · 2006 [cited by applicant]
US 7292722B2 · Lelescu et al. · 2007 [cited by applicant]
US 7623682B2 · Park et al. · 2009 [cited by applicant]
US 8217988B2 · Park · 2012 [cited by applicant]
US 8264524B1 · Davey · 2012 [cited by applicant]
US 11165958B2 · Kim · 2021 [cited by examiner]
US 20040105597A1 · Lelescu et al. · 2004 [cited by applicant]
US 20060034367A1 · Park · 2006 [cited by applicant]
US 20060034374A1 · Park et al. · 2006 [cited by applicant]
US 20060034529A1 · Park et al. · 2006 [cited by applicant]
US 20060210145A1 · Lee et al. · 2006 [cited by applicant]
US 20060251336A1 · Lelescu et al. · 2006 [cited by applicant]
US 20070133954A1 · Maeda et al. · 2007 [cited by applicant]
US 20110274166A1 · Jeon et al. · 2011 [cited by applicant]
US 20120057788A1 · Fukuhara et al. · 2012 [cited by applicant]
US 20120176367A1 · Genova et al. · 2012 [cited by applicant]
US 20130022113A1 · Chen et al. · 2013 [cited by applicant]
US 20130194255A1 · Lee et al. · 2013 [cited by applicant]
US 20130308709A1 · Norkin et al. · 2013 [cited by applicant]
US 20140146873A1 · Lee et al. · 2014 [cited by applicant]
US 20140184727A1 · Xiao et al. · 2014 [cited by applicant]
US 20140301463A1 · Rusanovskyy et al. · 2014 [cited by applicant]
US 20140307774A1 · Minoo et al. · 2014 [cited by applicant]
US 20140341549A1 · Hattori · 2014 [cited by applicant]
US 20150172544A1 · Deng et al. · 2015 [cited by applicant]
US 20150201204A1 · Chen et al. · 2015 [cited by applicant]
US 20160142697A1 · Budagavi et al. · 2016 [cited by applicant]
US 20160142740A1 · Sharman · 2016 [cited by applicant]
US 20160241837A1 · Cole et al. · 2016 [cited by applicant]
US 20160249056A1 · Tsukuba et al. · 2016 [cited by applicant]
US 20160328824A1 · Kim et al. · 2016 [cited by applicant]
US 20170336705A1 · Zhou et al. · 2017 [cited by applicant]
US 20170339391A1 · Zhou et al. · 2017 [cited by applicant]
US 20170347026A1 · Hannuksela · 2017 [cited by applicant]
US 20170366796A1 · Goldentouch · 2017 [cited by applicant]
US 20180025467A1 · Macmillan et al. · 2018 [cited by applicant]
US 20180027226A1 · Abbas et al. · 2018 [cited by applicant]
US 20180040164A1 · Newman et al. · 2018 [cited by applicant]
US 20180084257A1 · Abbas · 2018 [cited by applicant]
US 20180091735A1 · Wang et al. · 2018 [cited by applicant]
US 20190082161A1 · Cole et al. · 2019 [cited by applicant]
US 20190158849A1 · Yu et al. · 2019 [cited by applicant]
US 20190222862A1 · Shin et al. · 2019 [cited by applicant]
US 20200036997A1 · Li et al. · 2020 [cited by applicant]
US 20200322585A1 · Tsunashima · 2020 [cited by applicant]
US 20240048765A1 · Kim · 2024 [cited by applicant]
US 20240334062A1 · Kim · 2024 [cited by examiner]
US 20240340539A1 · Kim · 2024 [cited by examiner]
US 20240406565A1 · Kim · 2024 [cited by examiner]
CN 1174646A · 1998 [cited by applicant]
CN 1209933A · 1999 [cited by applicant]
CN 1759616A · 2006 [cited by applicant]
CN 101010960A · 2007 [cited by applicant]
CN 101479765A · 2009 [cited by applicant]
CN 100544444C · 2009 [cited by applicant]
CN 101641955A · 2010 [cited by applicant]
CN 101888555A · 2010 [cited by applicant]
CN 102611884A · 2012 [cited by applicant]
CN 102918840A · 2013 [cited by applicant]
CN 103250416A · 2013 [cited by applicant]
CN 103621084A · 2014 [cited by applicant]
CN 103907350A · 2014 [cited by applicant]
CN 104396259A · 2015 [cited by applicant]
CN 104584560A · 2015 [cited by applicant]
CN 104602013A · 2015 [cited by applicant]
CN 104662897A · 2015 [cited by applicant]
CN 104811637A · 2015 [cited by applicant]
CN 104833307A · 2015 [cited by applicant]
CN 105247870A · 2016 [cited by applicant]
CN 105554506A · 2016 [cited by applicant]
GB 2548358A · 2017 [cited by applicant]
JP 2015180040A · 2015 [cited by applicant]
KR 1020060015223A · 2006 [cited by applicant]
KR 1020060050350A · 2007 [cited by applicant]
KR 1020070103347A · 2007 [cited by applicant]
KR 1020110118527A · 2011 [cited by applicant]
KR 1020130088085A · 2013 [cited by applicant]
KR 1020150068299A · 2015 [cited by applicant]
KR 1020150113524A · 2015 [cited by applicant]
KR 1020160032909A · 2016 [cited by applicant]
WO WO2013159330A1 · 2013 [cited by applicant]
WO WO2014053518A1 · 2014 [cited by applicant]
WO WO2014084656A1 · 2014 [cited by applicant]
WO WO2015199478A1 · 2015 [cited by applicant]
WO WO2016026457A1 · 2016 [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video: Advanced Video Coding for Generic Audiovisual Services,” ITU-T Telecommunication Standardization Sector of ITU… [cited by applicant]
Fuldseth Arild, “Replacing Slices with Titles for High Level Parallelism,” Joint Collaborative Team on Video Coding (JCT-VC-D227) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Jan. 2011, (5 Pages in English). [cited by applicant]
Fuldseth Arild, “Tiles” Joint Collaborative Team on Video Coding (JCT-VC-F335) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Jul. 2011, (15 Pages in English). [cited by applicant]
Sullivan et al. “Overview of the High Efficiency Video Coding (HEVC) Standard”, [cited by applicant]
K. Misra, A. Segall, M. Horowitz, S. Xu, A. Fuldseth, & M. Zhou, “An Overview of Tiles in HEVC”, 7 IEEE J. of Selected Topics in Signal Processing 969-977 (Dec. 2013) (Year: 2013). [cited by applicant]
Choi., Byeongdoo et al., “WD on ISO/IEC 23000-20 Omnidirectional Media Application Format”, [cited by applicant]
Lu, International Organisation for Standardisation Organisation Internationale De Normalisation ISO/IEF JTC1/SC29/WG11 Coding of Moving Pictures and Audio, Tile Segmentation Scheme for OMAF, May-Jun. 2016, 4 pgs. [cited by applicant]
P. Hanhart, et al., “InterDigital's Response to the 360 Video Category in Joint Call for Evidence on Video Compression with Capability beyond Hevc”. [cited by applicant]
G. Auwera, et al., “AHG8: ACP with padding for 360-degree video”, [cited by applicant]
Y-K. Wang, et al., “On CMP padding and region-wise packing”, [cited by applicant]
International Search Report issued on Feb. 2, 2018 in counterpart International Patent Application No. PCT/KR2017/011143 (2 pages in English and 2 pages in Korean). [cited by applicant]
Korean Notice of Allowance, issued on Nov. 30, 2020 in counterpart Korean Patent Application No. 10-2019-7011755 (3 pages in Korean). [cited by applicant]
Chinese Office Action issued on Jan. 6, 2021 in counterpart Chinese Patent Application No. 201780073622.9 (9 pages in English and 8 pages in Chinese). [cited by applicant]
KIPO, Written Opinion for KR 10-2021-7034771 (Mar. 14, 2022) (Year: 2022). [cited by applicant]
Zhuo, Li, et al. “Quality Fine Granular Scalability Video Coding Method for Head Shoulder Sequence Images.” Acta Electonica Sinica 32.3 (2004): (5 pages in Japanese, Abstract in English). [cited by applicant]
Dong, P.Y., et al. “Re-loss-free intra Coding Using Integer Linear Programming for H.264/MPEG-4 AVC” Computer Science, vol. 36, 2009, (2 pages). [cited by applicant]
Kim, Jaejoon, et al. “On realization of modified encoding/decoding for high capacity panoramic video.” 2009 Fourth International Conference on Digital Information Management. IEEE, 2009, (Abstract). [cited by applicant]
Sullivan, Gary, et al. “Meeting report of the 23rd meeting of the Joint Collaborative Team on Video Coding (JCT-VC), San Diego, US, Feb. 19-26, 2016” JCTVC-W_Notes_d7, Joint Collaborative Team on Video Coding (JCT-VC) o… [cited by applicant]
ITU-T Recommendation H.365: High efficiency video coding (Apr. 2015) (Year: 2015). [cited by applicant]
Ohm, Jens-Rainer, et al. “Comparison of the coding efficiency of video coding standards- including high efficiency video coding (HEVC).” [cited by applicant]
Sze, Vivienne, et al. “High efficiency video coding (HEVC).” [cited by applicant]
Sjoberg, Rickard, et al. “Overview of HEVC high-level syntax and reference picture management.” [cited by applicant]
Bordes, Philippe, et al. “An overview of the emerging HEVC standard.” [cited by applicant]
Series, H. [cited by applicant]