IP Library › Granted Patent US 12,615,440
Granted Patent B2
US 12,615,440 · App. 18/447,068 · Granted Apr 28, 2026

Image data encoding/decoding method and apparatus

Inventor: Ki Baek Kim (Daejeon, KR)
Assignee: B1 INSTITUTE OF IMAGE TECHNOLOGY, INC.
H04N23/698G06T3/40H04N19/103H04N19/11H04N19/119H04N19/124H04N19/129H04N19/13H04N19/134H04N19/159H04N19/176H04N19/51H04N19/625
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,615,440
App. No.
18/447,068
Filed
Aug 9, 2023
Granted
Apr 28, 2026
Kind
B2
Art Unit
2488
USPC
375/240.016
Abstract

A method of decoding an image, includes obtaining at least one offset for a picture, deriving a variable for scaling for the picture based on the at least one offset, and performing inter prediction based on the variable for scaling for the picture. The at least one offset is defined with a direction of scaling.

Claims (72)

1 . A method of decoding a current image with a decoding apparatus, comprising:

receiving a bitstream including the current image;

dividing, based on coding information from the bitstream, a coding block in the current image to obtain one or more sub-blocks,

wherein the coding information includes a first flag indicating whether to divide the coding block to obtain one or more sub-blocks, a second flag indicating whether to divide the coding block asymmetrically, and a third flag indicating whether a division direction for the coding block is a horizontal direction or a vertical direction, the first flag, the second flag and the third flag being different from each other,

wherein the coding information further includes information on a division position of each of the one or more sub-blocks within the coding block, and

wherein, in case the coding block is divided asymmetrically, a first sub-block having a ¼ size of the coding block is obtained, the first sub-block belonging to the one or more sub-blocks;

obtaining inverse-quantized coefficients of the first sub-block by performing inverse-quantization on the first sub-block;

reconstructing a residual signal of the first sub-block by performing inverse-transform on the inverse-quantized coefficients; and

reconstructing the coding block based on the residual signal and a prediction signal corresponding to the residual signal,

wherein the prediction signal is generated by obtaining prediction information for the coding block from the bitstream and determining a prediction method for the coding block based on the prediction information, and performing prediction for the coding block based on the determined prediction information,

wherein the prediction method is determined as one among a plurality of prediction methods including an inter prediction method, and

wherein obtaining the inverse-quantized coefficients of the first sub-block comprises:

selecting one of a plurality of weight set candidates pre-defined in the decoding apparatus; and

performing the inverse-quantization on the first sub-block based on the selected weight set candidate.

2 . The method of claim 1 , wherein each of the weight set candidates includes a plurality of weight components, and

wherein at least one of the weight components belonging to the one of the weight set candidates is different from the weight components belonging to the other of the weight set candidates.

3 . The method of claim 1 , wherein a transform type for the inverse-transform of the inverse-quantized coefficients is determined by considering whether or not a size of the first sub-block is greater than a threshold size pre-defined in the decoding apparatus.

4 . The method of claim 3 , wherein a number of transform types available for the first sub-block when the size of the first sub-block is greater than the threshold size is different from the number of the transform types available for the first sub-block when the size of the first sub-block is less than or equal to the threshold size.

5 . The method of claim 4 , wherein a transform type for the inverse-transform of the quantized coefficients is determined by further considering at least one of the first flag, the second flag, or the third flag.

6 . A method of encoding a current image with an encoding apparatus, comprising:

dividing a coding block in the current image to obtain one or more sub-blocks;

obtaining transform coefficients of a first sub-block by performing transform on a residual signal of the first sub-block, the first sub-block belonging to the one or more sub-blocks;

obtaining quantized transform coefficients of the first sub-block by performing quantization on the transform coefficients of the first sub-block; and

generating a bitstream by encoding the quantized transform coefficients of the first sub-block,

wherein the residual signal is obtained based on a prediction signal corresponding to the residual signal,

wherein the prediction signal is generated by determining a prediction method for the coding block and performing prediction for coding block based on the determined prediction method,

wherein the determined prediction method is encoded into the bitstream using prediction information,

wherein the prediction method is determined as one among a plurality of prediction methods including an inter prediction method,

wherein coding information for dividing the coding block is encoded into the bitstream,

wherein the coding information includes a first flag indicating whether to divide the coding block to obtain the one or more sub-blocks, a second flag indicating whether to divide the coding block asymmetrically, and a third flag indicating whether a division direction for the coding block is a horizontal direction or a vertical direction, the first flag, the second flag and the third flag being different from each other,

wherein the coding information further includes information on a division position of each of the one or more sub-blocks within the coding block,

wherein, in case the coding block is divided asymmetrically, the first sub-block having a ¼ size of the coding block is obtained, and

wherein obtaining the quantized transform coefficients of the first sub-block comprises:

selecting one of a plurality of weight set candidates pre-defined in the encoding apparatus; and

performing the quantization on the transform coefficients of the first sub-block based on the selected weight set candidate.

7 . A non-transitory computer-readable medium for storing data associated with an image signal, comprising:

a data stream encoded by an encoding method,

wherein the encoding method comprises:

dividing a coding block in the current image to obtain one or more sub-blocks;

obtaining transform coefficients of a first sub-block by performing transform on a residual signal of the first sub-block, the first sub-block belonging to the one or more sub-blocks;

obtaining quantized transform coefficients of the first sub-block by performing quantization on the transform coefficients of the first sub-block; and

generating a bitstream by encoding the quantized transform coefficients of the first sub-block,

wherein the residual signal is obtained based on a prediction signal corresponding to the residual signal,

wherein the prediction signal is generated by determining a prediction method for the coding block and performing prediction for coding block based on the determined prediction method,

wherein the determined prediction method is encoded into the bitstream using prediction information,

wherein the prediction method is determined as one among a plurality of prediction methods including an inter prediction method,

wherein coding information for dividing the coding block is encoded into the bitstream,

wherein the coding information includes a first flag indicating whether to divide the coding block to obtain the one or more sub-blocks, a second flag indicating whether to divide the coding block asymmetrically, and a third flag indicating whether a division direction for the coding block is a horizontal direction or a vertical direction, the first flag, the second flag and the third flag being different from each other,

wherein the coding information further includes information on a division position of each of the one or more sub-blocks within the coding block,

wherein, in case the coding block is divided asymmetrically, the first sub-block having a ¼ size of the coding block is obtained, and

wherein obtaining the quantized transform coefficients of the first sub-block comprises:

selecting one of a plurality of weight set candidates pre-defined in the encoding apparatus; and

performing the quantization on the transform coefficients of the first sub-block based on the selected weight set candidate.

8 . A method for transmitting data associated with an image signal, the method comprising:

obtaining a data stream encoded by an encoding method; and

transmitting the data stream,

wherein the encoding method comprises:

dividing a coding block in the current image to obtain one or more sub-blocks;

obtaining transform coefficients of a first sub-block by performing transform on a residual signal of the first sub-block, the first sub-block belonging to the one or more sub-blocks;

obtaining quantized transform coefficients of the first sub-block by performing quantization on the transform coefficients of the first sub-block; and

generating a bitstream by encoding the quantized transform coefficients of the first sub-block,

wherein the residual signal is obtained based on a prediction signal corresponding to the residual signal,

wherein the prediction signal is generated by determining a prediction method for the coding block and performing prediction for coding block based on the determined prediction method,

wherein the determined prediction method is encoded into the bitstream using prediction information,

wherein the prediction method is determined as one among a plurality of prediction methods including an inter prediction method,

wherein coding information for dividing the coding block is encoded into the bitstream,

wherein the coding information includes a first flag indicating whether to divide the coding block to obtain the one or more sub-blocks, a second flag indicating whether to divide the coding block asymmetrically, and a third flag indicating whether a division direction for the coding block is a horizontal direction or a vertical direction, the first flag, the second flag and the third flag being different from each other,

wherein the coding information further includes information on a division position of each of the one or more sub-blocks within the coding block,

wherein, in case the coding block is divided asymmetrically, the first sub-block having a ¼ size of the coding block is obtained, and

wherein obtaining the quantized transform coefficients of the first sub-block comprises:

selecting one of a plurality of weight set candidates pre-defined in the encoding apparatus; and

performing the quantization on the transform coefficients of the first sub-block based on the selected weight set candidate.

Priority Claims (3)
KR 10-2016-0127893 · Oct 4, 2016 · national
KR 10-2016-0129391 · Oct 6, 2016 · national
KR 10-2017-0090621 · Jul 17, 2017 · national
Continuity (6)
Continuation 17512996 · Oct 28, 2021
Continuation 17211575 · Mar 24, 2021
Continuation 17085935 · Oct 30, 2020
Continuation 16372270 · Apr 1, 2019
Continuation PCTKR2017011149 · Oct 10, 2017
Related Publication 20230388647A1 · Nov 30, 2023
References Cited (176)
US 7623682B2 · Park et al. · 2009 [cited by applicant]
US 10009621B2 · Zhang et al. · 2018 [cited by applicant]
US 10104361B2 · Budagavi et al. · 2018 [cited by applicant]
US 20020141659A1 · Florin et al. · 2002 [cited by applicant]
US 20060034367A1 · Park · 2006 [cited by applicant]
US 20060034370A1 · Park et al. · 2006 [cited by applicant]
US 20060034374A1 · Park et al. · 2006 [cited by applicant]
US 20060034529A1 · Park et al. · 2006 [cited by applicant]
US 20060268991A1 · Segall et al. · 2006 [cited by applicant]
US 20070211797A1 · Kim et al. · 2007 [cited by applicant]
US 20080225945A1 · Wu et al. · 2008 [cited by applicant]
US 20080226123A1 · Birtwistle et al. · 2008 [cited by applicant]
US 20100086051A1 · Park et al. · 2010 [cited by applicant]
US 20100086052A1 · Park et al. · 2010 [cited by applicant]
US 20110134998A1 · Lee et al. · 2011 [cited by applicant]
US 20120082212A1 · Sadafale et al. · 2012 [cited by applicant]
US 20120082216A1 · Wang et al. · 2012 [cited by applicant]
US 20120114039A1 · Wang · 2012 [cited by applicant]
US 20120170648A1 · Chen et al. · 2012 [cited by applicant]
US 20120320970A1 · Drugeon et al. · 2012 [cited by applicant]
US 20130089265A1 · Yie et al. · 2013 [cited by applicant]
US 20130108182A1 · Yie et al. · 2013 [cited by applicant]
US 20130128971A1 · Guo et al. · 2013 [cited by applicant]
US 20130128974A1 · Chien et al. · 2013 [cited by applicant]
US 20130188704A1 · Zhou · 2013 [cited by applicant]
US 20130315300A1 · Lee et al. · 2013 [cited by applicant]
US 20130322525A1 · Tanaka · 2013 [cited by examiner]
US 20140168362A1 · Hannuksela et al. · 2014 [cited by applicant]
US 20140254679A1 · Ramasubramonian et al. · 2014 [cited by applicant]
US 20140301483A1 · Wang et al. · 2014 [cited by applicant]
US 20140355666A1 · Zhang et al. · 2014 [cited by applicant]
US 20140376635A1 · Park et al. · 2014 [cited by applicant]
US 20150172688A1 · Maeda et al. · 2015 [cited by applicant]
US 20150201204A1 · Chen et al. · 2015 [cited by applicant]
US 20150229967A1 · Lee · 2015 [cited by applicant]
US 20150237351A1 · Lee et al. · 2015 [cited by applicant]
US 20150312588A1 · Yamamoto · 2015 [cited by examiner]
US 20150350659A1 · Auyeung et al. · 2015 [cited by applicant]
US 20160012855A1 · Krishnan · 2016 [cited by applicant]
US 20160057433A1 · Kudo et al. · 2016 [cited by applicant]
US 20160057440A1 · Jeong et al. · 2016 [cited by applicant]
US 20160142697A1 · Budagavi et al. · 2016 [cited by applicant]
US 20160219290A1 · Zhao · 2016 [cited by examiner]
US 20160227214A1 · Rapaka · 2016 [cited by examiner]
US 20160249056A1 · Tsukuba et al. · 2016 [cited by applicant]
US 20160328824A1 · Kim et al. · 2016 [cited by applicant]
US 20170078665A1 · Zheng · 2017 [cited by applicant]
US 20170150186A1 · Zhang · 2017 [cited by examiner]
US 20170214937A1 · Lin et al. · 2017 [cited by applicant]
US 20170244964A1 · Liu · 2017 [cited by examiner]
US 20170347026A1 · Hannuksela · 2017 [cited by applicant]
US 20180041764A1 · Lin et al. · 2018 [cited by applicant]
US 20190052859A1 · Budagavi et al. · 2019 [cited by applicant]
US 20190215532A1 · He et al. · 2019 [cited by applicant]
US 20190222862A1 · Shin et al. · 2019 [cited by applicant]
US 20190260990A1 · Lim et al. · 2019 [cited by applicant]
CN 1256043A · 2000 [cited by applicant]
CN 1870748A · 2006 [cited by applicant]
CN 101002479A · 2007 [cited by applicant]
CN 101010960A · 2007 [cited by applicant]
CN 101047795A · 2007 [cited by applicant]
CN 101990104A · 2011 [cited by applicant]
CN 102239694A · 2011 [cited by applicant]
CN 102246525A · 2011 [cited by applicant]
CN 102893608A · 2013 [cited by applicant]
CN 102948149A · 2013 [cited by applicant]
CN 103004197A · 2013 [cited by applicant]
CN 103079067A · 2013 [cited by applicant]
CN 103119940A · 2013 [cited by applicant]
CN 103535039A · 2014 [cited by applicant]
CN 104012089A · 2014 [cited by applicant]
CN 104350745A · 2015 [cited by applicant]
CN 104685872A · 2015 [cited by applicant]
CN 104718756A · 2015 [cited by applicant]
CN 104737540A · 2015 [cited by applicant]
CN 104885470A · 2015 [cited by applicant]
CN 104935945A · 2015 [cited by applicant]
CN 104969553A · 2015 [cited by applicant]
CN 105324984A · 2016 [cited by applicant]
CN 105392008A · 2016 [cited by applicant]
CN 105554506A · 2016 [cited by applicant]
CN 105794208A · 2016 [cited by applicant]
CN 105850133A · 2016 [cited by applicant]
EP 2991356A1 · 2016 [cited by applicant]
EP 3434018B1 · 2019 [cited by applicant]
FR 2811112A1 · 2002 [cited by applicant]
JP 2012110054A · 2012 [cited by applicant]
JP 2013098711A · 2013 [cited by applicant]
JP 2013118424A · 2013 [cited by applicant]
JP 2013229674A · 2013 [cited by applicant]
JP 2014519764A · 2014 [cited by applicant]
JP 2014523702A · 2014 [cited by applicant]
JP 2014535221A · 2014 [cited by applicant]
JP 2015035641A · 2015 [cited by applicant]
JP 2015521442A · 2015 [cited by applicant]
JP 2016531467A · 2016 [cited by applicant]
JP 2017112639A · 2017 [cited by applicant]
JP 2019525563A · 2019 [cited by applicant]
JP 2019536314A · 2019 [cited by applicant]
JP 7201776B2 · 2023 [cited by applicant]
JP 7224280B2 · 2023 [cited by applicant]
JP 202340182A · 2023 [cited by applicant]
JP 2023040179A · 2023 [cited by applicant]
JP 7269455B2 · 2023 [cited by applicant]
JP 7269456B2 · 2023 [cited by applicant]
JP 2023066544A · 2023 [cited by applicant]
JP 7353522B2 · 2023 [cited by applicant]
JP 2023181932A · 2023 [cited by applicant]
JP 2023181961A · 2023 [cited by applicant]
JP 2024026707A · 2024 [cited by applicant]
JP 202442822A · 2024 [cited by applicant]
JP 202442827A · 2024 [cited by applicant]
JP 2024109865A · 2024 [cited by applicant]
KR 1020060050350A · 2007 [cited by applicant]
KR 1020070103347A · 2007 [cited by applicant]
KR 1020140019221A · 2014 [cited by applicant]
KR 1020150068299A · 2015 [cited by applicant]
KR 20150070845A · 2015 [cited by applicant]
KR 1020150083444A · 2015 [cited by applicant]
KR 20150083444A · 2015 [cited by applicant]
KR 1020160032909A · 2016 [cited by applicant]
KR 1020160097182A · 2016 [cited by applicant]
WO 2009003885A2 · 2009 [cited by applicant]
WO 2012023817A2 · 2012 [cited by applicant]
WO 2012081879A1 · 2012 [cited by applicant]
WO 2012176387A1 · 2012 [cited by applicant]
WO 2013023005A1 · 2013 [cited by applicant]
WO 2013047805A1 · 2013 [cited by applicant]
WO WO2014005297A1 · 2014 [cited by applicant]
WO 2014120369A1 · 2014 [cited by applicant]
WO 2014203805A1 · 2014 [cited by applicant]
WO 2015100732A1 · 2015 [cited by applicant]
WO 2015143346A1 · 2015 [cited by applicant]
WO WO2015199478A1 · 2015 [cited by applicant]
WO WO2016026457A1 · 2016 [cited by applicant]
WO 2016053494A1 · 2016 [cited by applicant]
WO WO2016056822A1 · 2016 [cited by applicant]
WO 2016090568A1 · 2016 [cited by applicant]
WO 2017123980A1 · 2017 [cited by applicant]
WO 2017162912A1 · 2017 [cited by applicant]
WO 2017165375A1 · 2017 [cited by applicant]
WO 2017165509A1 · 2017 [cited by applicant]
WO WO2017211294A1 · 2017 [cited by applicant]
WO 2018002425A2 · 2018 [cited by applicant]
WO 2018009746A1 · 2018 [cited by applicant]
WO 2018066988A1 · 2018 [cited by applicant]
WO 2018066991A1 · 2018 [cited by applicant]
WO 2019026807A1 · 2019 [cited by applicant]
WO 2019059679A1 · 2019 [cited by applicant]
Huang, H., et al. “EE2. 1: Quadtree plus binary tree structure integration with JEM tools: JVET-C0024_r1, Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO.” IEC JTC1/SC29/WG11 3rd Meeting: Geneva, CH, May 2… [cited by applicant]
Kawamura, Kei, et al. “Extended deblocking-filter process for large block boundary” JVET-C0049_r2, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 3rd Meeting: Geneva, CH, May 26-J… [cited by applicant]
Zhao, Xin, et al. “Grouped signalling for transform in QTBT” JVET-C0054, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 3rd Meeting: Geneva, CH, May 26-Jun. 1, 2016, (3 pages). [cited by applicant]
Le Léannec, F., et al. “Asymmetric Coding Units in QTBT” JVET-D0064, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016, (10 pages). [cited by applicant]
Li, Xiang, et al. “Multi-Type-Tree” JVET-D0117r1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016, (3 pages). [cited by applicant]
Kim et al. “AHG4: Constraint for slice and tile”, [cited by applicant]
Boyce et al. “Draft high efficiency video coding (HEVC) version 2, combined format range extensions (RExt), scalability (SHVC), and multi-view (MV-HEVC) extensions.” [cited by applicant]
Huang et al. “EE2. 1: Quadtree plus binary tree structure Integration with JEM tools.” [cited by applicant]
Li et al. “Multi-Type-Tree” [cited by applicant]
Budagava et al. “OMAF: Metadata for guard margin at polygon boundaries in 360 degrees video to prevent boundary artifacts ” [cited by applicant]
He et al. “AHG8: Geometry padding for 360 video coding.” [cited by applicant]
Chen et al. “Algorithm Description of Joint Exploration Test Model 7 (JEM 7)”, [cited by applicant]
International Search Report issued on Feb. 6, 2018 in counterpart Korean Patent Application PCT/KR2017/011149 (9 Pages). [cited by applicant]
Non-Final Office Action issued on Mar. 6, 2020 in counterpart U.S. Appl. No. 16/372,270 (7 pages). [cited by applicant]
Quadtree Based Nonsquare Block Structure for Inter Frame Coding in High Efficiency Video Coding, Yuan Yuan, et al. IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, No. 12, Dec. 2012. [cited by applicant]
Zhichu He et al., “AVS2-video coding standard—An application-oriented and high performance video coding standard,” IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Jul. 14, 2014. [cited by applicant]
Yoav Cohen, et al. “Hierarchical Coding of Binary Images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-7, No. 3, May 1985. [cited by applicant]
Yuwen He, et al. “AHG8: Geometry padding for 360 video coding,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-D0075, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016. [cited by applicant]
Yuwen He, et al. “AHG8: Geometry padding for 360 video coding,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016. [cited by applicant]
Huang, et al., “EE2.1 Quadtree plus binary tree structure integration with JEM tools,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11, 3rd Meeting: Geneva, CH, May 26-Jun. 1, 2016, … [cited by applicant]
Zhichu He, et al. “AVS2-video coding standard—An application-oriented and high performance video coding standard,” 2014 IEEE International Conference on Multimedia and Expo Workshop (ICMEW), Jul. 14-18, 2014. [cited by applicant]
Cui Yubin, “Research on Key Technologies of Wireless Video Communication Systems Based on H.264/AVC,” Ph.D. Thesis, Beijing University of Posts and Telecommunications, School of Information and Communication Engineering… [cited by applicant]
Byeongdoo Choi, et al., “WD on ISO/IEC 23000-20 Omnidirectional Media Application Format,” ISO/IEC JTC 1/SC 29/WG 11. ISO/IEC JTC1/SC29/WG11 N16189, Jun. 2016, Geneva, Switzerland. [cited by applicant]
Kiran Misra, “An Overview of Tiles in HEVC,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, Issue: 6, Dec. 2013. [cited by applicant]
Yoav Cohen, “Hierarchical Coding of Binary Images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-7, No. 3, May 1985. [cited by applicant]
Minhua Zhou, “AHG4: Enable parallel decoding with tiles,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 10th Meeting, Stockholm, Sweden, Jul. 11-20, 2012, JCTVC-J0088. [cited by applicant]
Benjamin Bross, et al. “High Efficiency Video Coding (HEVC) text specification draft 10 (for FDIS & Last Call),” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG 11 , 12th Mee… [cited by applicant]