IP Library Granted Patent US 12,301,820
Granted Patent B2
US 12,301,820 · App. 18/532,387 · Granted May 13, 2025

Transform-based image coding method and device therefor

Inventors: Moonmo Koo (Seoul, KR); Jaehyun Lim (Seoul, KR); Junghak Nam (Seoul, KR); Seunghwan Kim (Seoul, KR)
Assignee: BEIJING XIAOMI MOBILE SOFTWARE CO., LTD.
H04N19/132H04N19/105H04N19/176H04N19/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,301,820
App. No.
18/532,387
Granted
May 13, 2025
Kind
B2
Abstract

An image decoding method, according to the present document, may comprise the steps of: deriving transform coefficients for a current block on the basis of residual information; determining whether a significant coefficient is present in a second region excluding a first region in the top-left end of the current block; parsing a LFNST index from a bitstream if the significant coefficient is not present in the second region; deriving modified transform coefficients by applying a LFNST matrix, derived on the basis of the LFNST index, to transform coefficients of the first region; and deriving residual samples of the current block on the basis of an inverse primary transform of the modified transform coefficients.

Claims (52)

1. An image decoding method performed by a decoding apparatus, the image decoding method comprising:

deriving transform coefficients for a current block on the basis of residual information;

determining whether a significant coefficient exists in a second region, the second region being a region other than a first region including a top-left sample position of the current block;

parsing a Low Frequency Non-Separable Transform (LFNST) index from a bitstream on the basis of the determination that the significant coefficient does not exist in the second region;

deriving prediction samples for the current block;

deriving modified transform coefficients by applying an LFNST matrix derived on the basis of the LFNST index to transform coefficients in the first region;

deriving residual samples for the current block on the basis of an inverse primary transform for the modified transform coefficients; and

generating a reconstructed picture on the basis of the residual samples and the prediction samples for the current block,

wherein on the basis of a size of the current block being 4×4, the first region to which the LFNST matrix is applied is from the top-left sample position of the current block to an 8th sample position in a scan order,

wherein on the basis of the size of the current block being 8×8, the first region to which the LFNST matrix is applied is from the top-left sample position of the current block to the 8th sample position in the scan order,

wherein on the basis of one of the width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, the first region to which the LFNST matrix is applied is a top-left 4×4 area of the current block, and

wherein on the basis of the size of the current block being greater than 8×8, the first region to which the LFNST matrix is applied is a top-left 4×4 area of the current block.

2. The image decoding method of claim 1 , wherein a predetermined number of the modified transform coefficients are derived on the basis of the size of the current block.

3. The image decoding method of claim 2 , wherein on the basis of the one of the width or height of the current block being greater than or equal to 4 and the other of the width or height of the current block being equal to 4, 16 modified transform coefficients are derived.

4. The image decoding method of claim 3 , wherein the 16 modified transform coefficients are arranged in the top-left 4×4 area of the current block.

5. The image decoding method of claim 3 , wherein the 16 modified transform coefficients are arranged in a vertical or horizontal order according to an intra prediction mode of the current block.

6. The image decoding method of claim 3 , wherein on the basis of the size of the current block being greater than or equal to 8×8, 48 modified transform coefficients are derived.

7. An image encoding method performed by an image encoding apparatus, the image encoding method comprising:

deriving prediction samples for a current block;

deriving residual samples for the current block on the basis of the prediction samples;

deriving transform coefficients for the current block on the basis of a primary transform for the residual samples;

deriving modified transform coefficients for the current block on the basis of transform coefficients of a first region including a top-left sample position of the current block and a predetermined Low Frequency Non-Separable Transform (LFNST) matrix; and

encoding residual information related to the modified transform coefficients and an LFNST index indicating the LFNST matrix,

wherein on the basis of a size of the current block being 4×4, the first region to which the LFNST matrix is applied is the current block,

wherein on the basis of the size of the current block being 8×8, the first region to which the LFNST matrix is applied is top-left, top-right, and bottom-left 4×4 areas of the current block,

wherein on the basis of one of a width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, the first region to which the LFNST matrix is applied is a top-left 4×4 area of the current block, and

wherein on the basis of the size of the current block being greater than 8×8, the first region to which the LFNST matrix is applied is the top-left 4×4 area, a right 4×4 area neighboring to the top-left 4×4 area, and a bottom 4×4 area neighboring to the top-left 4×4 area of the current block.

8. The image encoding method of claim 7 , wherein the transform coefficients within the first region are one-dimensionally arranged in a vertical or horizontal order according to an intra prediction mode of the current block for a multiplication operation with the LFNST matrix.

9. The image encoding method of claim 7 , wherein a predetermined number of the modified transform coefficients are derived on the basis of the size of the current block.

10. The image encoding method of claim 9 , wherein on the basis of the size of the current block being 4×4 or 8×8, 8 modified transform coefficients are derived.

11. The image encoding method of claim 9 , wherein on the basis of the one of the width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, 16 modified transform coefficients are derived.

12. The image encoding method of claim 9 , wherein on the basis of the size of the current block being greater than 8×8, 16 modified transform coefficients are derived.

13. The image encoding method of claim 9 , wherein on the basis of the size of the current block being 4×4 or 8×8, the modified transform coefficients are arranged from the top-left sample position of the current block to the 8th sample position in a scan order,

wherein on the basis of the one of the width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, the modified transform coefficients are arranged in the top-left 4×4 area of the current block in the scan order, and

wherein on the basis of the size of the current block being greater than 8×8, the modified transform coefficients are arranged in the top-left 4×4 area of the current block in the scan order.

14. A non-transitory computer-readable digital storage medium that stores a bitstream generated by an image encoding method, the image encoding method comprising:

deriving prediction samples for a current block;

deriving residual samples for the current block on the basis of the prediction samples;

deriving transform coefficients for the current block on the basis of a primary transform for the residual samples;

deriving modified transform coefficients for the current block on the basis of transform coefficients of a first region including a top-left sample position of the current block and a predetermined Low Frequency Non-Separable Transform (LFNST) matrix; and

encoding residual information related to the modified transform coefficients and an LFNST index indicating the LFNST matrix to generate the bitstream,

wherein on the basis of a size of the current block being 4×4, the first region to which the LFNST matrix is applied is the current block,

wherein on the basis of the size of the current block being 8×8, the first region to which the LFNST matrix is applied is top-left, top-right, and bottom-left 4×4 areas of the current block,

wherein on the basis of one of a width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, the first region to which the LFNST matrix is applied is a top-left 4×4 area of the current block, and

wherein on the basis of the size of the current block being greater than 8×8, the first region to which the LFNST matrix is applied is the top-left 4×4 area, a right 4×4 area neighboring to the top-left 4×4 area, and a bottom 4×4 area neighboring to the top-left 4×4 area of the current block.

15. A transmission method, comprising:

obtaining a bitstream, wherein the bitstream is generated by: deriving prediction samples for a current block, deriving residual samples for the current block on the basis of the prediction samples, deriving transform coefficients for the current block on the basis of a primary transform for the residual samples, deriving modified transform coefficients for the current block on the basis of transform coefficients of a first region including a top-left sample position of the current block and a predetermined Low Frequency Non-Separable Transform (LFNST) matrix, and encoding residual information related to the modified transform coefficients and an LFNST index indicating the LFNST matrix; and

transmitting the bitstream,

wherein on the basis of a size of the current block being 4×4, the first region to which the LFNST matrix is applied is the current block,

wherein on the basis of the size of the current block being 8×8, the first region is top-left, top-right, and bottom-left 4×4 areas of the current block,

wherein on the basis of one of a width or height of the current block being greater than 4 and the other of the width or height of the current block being 4, the first region to which the LFNST matrix is applied is a top-left 4×4 area of the current block, and

wherein on the basis of the size of the current block being greater than 8×8, the first region to which the LFNST matrix is applied is the top-left 4×4 area, a right 4×4 area neighboring to the top-left 4×4 area, and a bottom 4×4 area neighboring to the top-left 4×4 area of the current block.

Assignments (2)
NUNC PRO TUNC ASSIGNMENT Recorded Jan 22, 2025
From: LG ELECTRONICS INC.
To: BEIJING XIAOMI MOBILE SOFTWARE CO., LTD.
Reel/Frame 069988/0424 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2023
From: KOO, MOONMO; LIM, JAEHYUN; NAM, JUNGHAK; KIM, SEUNGHWAN
To: LG ELECTRONICS INC.
Reel/Frame 065925/0899 →
Continuity (6)
Continuation 17894745 · Aug 24, 2022
Continuation 17535090 · Nov 24, 2021
Continuation PCTKR2020007992 · Jun 19, 2020
Provisional Application 62865133 · Jun 21, 2019
Provisional Application 62863833 · Jun 19, 2019
Related Publication 20240129476A1 · Apr 18, 2024
References Cited (11)
US 20190281321A1 · Zhao · 2019 [cited by examiner]
US 20200021810A1 · Li · 2020 [cited by examiner]
US 20210306666A1 · Lee · 2021 [cited by examiner]
JP 2018530247A · 2018 [cited by applicant]
WO WO2018174402A1 · 2018 [cited by examiner]
Koo “Description of SDR Video Coding Technology Proposal by LG Electronics” JVET-J0017-v1, San Diego CA, Apr. 10-20, 2018. (Year: 2018). [cited by examiner]
Chen et al., “Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5),” JVET-N1002-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. … [cited by applicant]
Koo et al., “Reduced Secondary Transform (RST) Algorithm Description,” JVET-N0193, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 16 pages. [cited by applicant]
Office Action in Japanese Appln. No. 2021-573861, mailed on Jun. 21, 2024, 9 pages (with English translation). [cited by applicant]
Office Action in Korean Appln. No. 10-2021-7032851, mailed on Jun. 12, 2024, 13 pages (with English translation). [cited by applicant]
Siekmann et al., “CE6-related: Simplification of the Reduced Secondary Transform,” JVET-N0555-v3, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, … [cited by applicant]