IP Library Granted Patent US 12,666,048
Granted Patent B2
US 12,666,048 · App. 18/829,826 · Granted Jun 23, 2026

Matrix-based intra prediction device and method

Inventors: Mehdi Salehifar (Seoul, KR); Seunghwan Kim (Seoul, KR)
Assignee: NOKIA TECHNOLOGIES OY
H04N19/159H04N19/132H04N19/176H04N19/593H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,666,048
App. No.
18/829,826
Filed
Sep 10, 2024
Granted
Jun 23, 2026
Kind
B2
Art Unit
2485
USPC
375/240.12
Abstract

An image decoding method according to the present document can comprise the steps of: on the basis of first flag information indicating whether or not matrix-based intra prediction (MIP) can be applied to a current block, receiving second flag information indicating whether or not the MIP is used for the current block; receiving matrix-based intra prediction (MIP) mode information on the basis of the second flag information; generating intra prediction samples for the current block on the basis of the MIP mode information; and generating reconstructed samples for the current block on the basis of the intra prediction samples.

Claims (38)

1 . An image decoding method performed by a decoding apparatus, the method comprising:

receiving residual information;

receiving second flag information related to whether an intra prediction type for a current block is matrix-based intra prediction (MIP) based on first flag information related to whether the MIP is enabled;

receiving MIP mode information based on the second flag information;

generating intra prediction samples for the current block based on the MIP mode information;

deriving quantized transform coefficients based on the residual information;

deriving transform coefficients based on the quantized transform coefficients;

deriving residual samples based on the transform coefficients; and

generating reconstructed samples for the current block based on the intra prediction samples and the residual samples,

wherein the MIP mode information is index information related to an MIP matrix for the current block,

wherein the MIP matrix is derived based on a size of the current block and the index information,

wherein the intra prediction samples is generated based on the MIP matrix,

wherein a syntax element bin string for the MIP mode information is binarized by a TB (truncated binarization) method,

wherein a maximum length of the syntax element bin string is set to different three values based on the size of the current block, and

wherein based on a width and a height of the current block being 4, the maximum length has a largest value.

2 . An image encoding method performed by an encoding apparatus, the method comprising:

deriving whether matrix-based intra prediction (MIP) is applied to a current block;

deriving intra prediction samples for the current block based on that the MIP is applied to the current block;

deriving residual samples for the current block based on the intra prediction samples;

deriving transform coefficients based on the residual samples;

deriving quantized transform coefficients based on the transform coefficients;

generating residual information based on the quantized transform coefficients; and

generating a bitstream by encoding the residual information and information on the MIP,

wherein the information on the MIP comprises first flag information related to whether the MIP is enabled, second flag information related to whether an intra prediction type for the current block is the MIP and MIP mode information which is index information related to an MIP matrix for the current block,

wherein the MIP matrix is derived based on a size of the current block and the index information,

wherein the intra prediction samples is derived based on the MIP matrix,

wherein a syntax element bin string for the MIP mode information is binarized by a TB (truncated binarization) method,

wherein a maximum length of the syntax element bin string is set to different three values based on the size of the current block, and

wherein based on a width and a height of the current block being 4, the maximum length has a largest value.

3 . A transmitting method of data for an image, the method comprising:

obtaining a bitstream for the image, wherein the bitstream is generated based on deriving whether matrix-based intra prediction (MIP) is applied to a current block, deriving intra prediction samples for the current block based on that the MIP is applied to the current block, deriving residual samples for the current block based on the intra prediction samples, deriving transform coefficients based on the residual samples, deriving quantized transform coefficients based on the transform coefficients, generating residual information based on the quantized transform coefficients, and generating a bitstream by encoding the residual information and information on the MIP; and

transmitting the data comprising the bitstream,

wherein the information on the MIP comprises first flag information related to whether the MIP is enabled, second flag information related to whether an intra prediction type for the current block is the MIP and MIP mode information which is index information related to an MIP matrix for the current block,

wherein the MIP matrix is derived based on a size of the current block and the index information,

wherein the intra prediction samples is derived based on the MIP matrix,

wherein a syntax element bin string for the MIP mode information is binarized by a TB (truncated binarization) method,

wherein a maximum length of the syntax element bin string is set to different three values based on the size of the current block, and

wherein based on a width and a height of the current block being 4, the maximum length has a largest value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 9, 2025
From: LG ELECTRONICS INC.
To: NOKIA TECHNOLOGIES OY
Reel/Frame 072859/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2024
From: SALEHIFAR, MEHDI; KIM, SEUNGHWAN
To: LG ELECTRONICS INC.
Reel/Frame 068542/0625 →
Continuity (6)
Continuation 18385758 · Oct 31, 2023
Continuation 18094732 · Jan 9, 2023
Continuation 17527951 · Nov 16, 2021
Continuation PCTKR2020007221 · Jun 3, 2020
Provisional Application 62856715 · Jun 3, 2019
Related Publication 20240430447A1 · Dec 26, 2024
References Cited (88)
US 8194749B2 · Kim et al. · 2012 [cited by applicant]
US 20130114695A1 · Joshi · 2013 [cited by examiner]
US 20130188691A1 · Haque et al. · 2013 [cited by applicant]
US 20140241436A1 · Laroche et al. · 2014 [cited by applicant]
US 20150264348A1 · Zou · 2015 [cited by examiner]
US 20170374366A1 · Xiu et al. · 2017 [cited by applicant]
US 20180048896A1 · Park et al. · 2018 [cited by applicant]
US 20180255304A1 · Jeon et al. · 2018 [cited by applicant]
US 20180332312A1 · Liu et al. · 2018 [cited by applicant]
US 20200314429A1 · Haque et al. · 2020 [cited by applicant]
US 20200322620A1 · Zhao et al. · 2020 [cited by applicant]
US 20200344468A1 · Lin · 2020 [cited by examiner]
US 20200359050A1 · Van Der Auwera · 2020 [cited by examiner]
US 20210168362A1 · Jun et al. · 2021 [cited by applicant]
US 20210195176A1 · Kim · 2021 [cited by examiner]
US 20210321090A1 · Deng · 2021 [cited by examiner]
US 20220078450A1 · Salehifar et al. · 2022 [cited by applicant]
CN 107431817A · 2017 [cited by applicant]
CN 109792521A · 2019 [cited by applicant]
GB 2495990A · 2013 [cited by applicant]
IN 201817017885 · 2018 [cited by applicant]
JP 2014147111A · 2014 [cited by applicant]
JP 7379543B2 · 2023 [cited by applicant]
JP 7509978B2 · 2024 [cited by applicant]
KR 1020110044145A · 2011 [cited by applicant]
KR 1020170131473A · 2017 [cited by applicant]
WO 2017191782A1 · 2017 [cited by applicant]
WO 2019039806A1 · 2019 [cited by applicant]
WO 2020205705A1 · 2020 [cited by applicant]
WO WO2020207500A1 · 2020 [cited by examiner]
WO WO2020211765A1 · 2020 [cited by examiner]
WO WO2020239017A1 · 2020 [cited by examiner]
WO WO2020239018A1 · 2020 [cited by examiner]
“Multiple Linear Regression for High Efficiency Video Intra Coding”—Zhang et al., 978-1-5386-4658-8/18/$31.00 A © 2019 IEEE (Year: 2019). [cited by examiner]
Advisory Action (PTOL-303) Mailed on Jul. 25, 2022 for U.S. Appl. No. 17/527,951, 3 page(s). [cited by applicant]
B. Bross et al., “Versatile Video Coding {Draft 5)”, Joint Video Experts Team {JVET) of ITU-T SG 16 WP 3 and ISO/EC JTC 1/SC 29/WG 11, Mar. 19-27, 2019, JVET-N1001-v7. [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 4)”, Editors, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC Jtc 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, Document: JVET-M1001-v… [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 4)”, Editors, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, Document: JVET-M1001-v… [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 5)”, Editors, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, Document: JVET-N1001-v8,… [cited by applicant]
Benjamin Bross, et al., “Versatile Video Coding (Draft 6)”. Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019. JVET-02001-v9. [cited by applicant]
Decision of Refusal for Japanese Application No. 2021-571622 dated May 9, 2023, 8 pages. [cited by applicant]
Decision to Grant for Chinese Application No. 202080044821.9 dated Jul. 31, 2024, 3 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2021-571622 dated Oct. 10, 2023, 5 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2023-146224 dated May 21, 2024, 5 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2024-099927 dated May 20, 2025, 5 pages. [cited by applicant]
Decision to Grant for Korean Application No. 10-2021-7035516 dated Nov. 3, 2025, 6 pages. [cited by applicant]
Decision to Refuse for European Application No. 20818136.2 dated May 15, 2024, 11 pages. [cited by applicant]
Extended European Search Report for European Application No. 2081813.6 dated Jun. 29, 2022, 11 pages. [cited by applicant]
Final Rejection Mailed on Apr. 25, 2022 for U.S. Appl. No. 17/527,951, 15 page(s). [cited by applicant]
G. Sullivan et al., “Meeting Report of the 15th Meeting of the Joint Video Experts Team (JVET), Gothenburg, SE, 3-12 Uuly 2019”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1SO/IEC JTC 1/SC 29/WG 11, Jul. 3-… [cited by applicant]
IPEA/409—International Preliminary Report on Patentability Mailed on Dec. 16, 2021 for WO Application No. PCT/KR20/007221, 10 page(s). [cited by applicant]
J. Chen et al., “Algorithm description for Versatile Video Coding and Test Model 5 {VTM 5)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Mar. 19-27, 2019, JVET-N1002-v1. [cited by applicant]
J. Choi et al., “Non-CE3: MIP mode reduction”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 3-12, 2019, JVET-O0397. [cited by applicant]
J. Choi et al., “Non-CE3: Simplified mode mapping in MIP”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 3-12, 2019, JVET-O0396-v2. [cited by applicant]
J. Pfaff, et al., “Non-CE3: Simplifications of MIP”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 3-12, 2019, JVET-O0925-r1. [cited by applicant]
Jonathan Pfaff, et al., “CE3: Affine linear weighted intra prediction (CE3-4.1, CE3-4.2)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019. J… [cited by applicant]
M. Salehifar et al., “CE3 Related: Low Memory and Computational Complexity Matrix Based Intra Prediction (MIP)”, Uoint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1SO/IEC JTC 1/SC 29/WG 11, Jul. 3-12, 2019, JVET-0… [cited by applicant]
Medhi Salehifar et al., “CE3 Related: No. MPM Derivation for Matrix Based Intra Prediction (MIP)”, LG Electronics, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenb… [cited by applicant]
Minutes of the Oral Proceedings for European Application No. 20818136.2 dated May 13, 2024, 4 pages. [cited by applicant]
Non-Final Rejection Mailed on Jan. 19, 2022 for U.S. Appl. No. 17/527,951, 14 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Aug. 2, 2023 for U.S. Appl. No. 18/094,732, 9 page (s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Jan. 12, 2023 for U.S. Appl. No. 17/527,951, 2 page (s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Jul. 8, 2024 for U.S. Appl. No. 18/385,758, 2 page (s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Jun. 11, 2024 for U.S. Appl. No. 18/385,758, 9 page (s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Oct. 7, 2022 for U.S. Appl. No. 17/527,951, 8 page (s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Sep. 5, 2024 for U.S. Appl. No. 18/385,758, 2 page (s). [cited by applicant]
Notice of Final Rejection for Korean Application No. 10-2021-7035516 dated Feb. 20, 2025, 6 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2021-571622 dated Dec. 20, 2022, 8 pages. [cited by applicant]
Office Action Appendix Mailed on Apr. 25, 2022 for U.S. Appl. No. 17/527,951, 1 page(s). [cited by applicant]
Office Action Appendix Mailed on Apr. 25, 2022 for U.S. Appl. No. 17/527,951, 10 page(s). [cited by applicant]
Office Action Appendix Mailed on Apr. 25, 2022 for U.S. Appl. No. 17/527,951, 2 page(s). [cited by applicant]
Office Action Appendix Mailed on Aug. 2, 2023 for U.S. Appl. No. 18/094,732, 1 page(s). [cited by applicant]
Office Action Appendix Mailed on Aug. 2, 2023 for U.S. Appl. No. 18/094,732, 2 page(s). [cited by applicant]
Office Action Appendix Mailed on Aug. 2, 2023 for U.S. Appl. No. 18/094,732, 9 page(s). [cited by applicant]
Office Action Appendix Mailed on Jan. 19, 2022 for U.S. Appl. No. 17/527,951, 1 page(s). [cited by applicant]
Office Action Appendix Mailed on Jan. 19, 2022 for U.S. Appl. No. 17/527,951, 2 page(s). [cited by applicant]
Office Action Appendix Mailed on Jan. 19, 2022 for U.S. Appl. No. 17/527,951, 9 page(s). [cited by applicant]
Office Action Appendix Mailed on Jun. 11, 2024 for U.S. Appl. No. 18/385,758, 9 page(s). [cited by applicant]
Office Action Appendix Mailed on Oct. 7, 2022 for U.S. Appl. No. 17/527,951, 1 page(s). [cited by applicant]
Office Action Appendix Mailed on Oct. 7, 2022 for U.S. Appl. No. 17/527,951, 10 page(s). [cited by applicant]
Office Action Appendix Mailed on Oct. 7, 2022 for U.S. Appl. No. 17/527,951, 2 page(s). [cited by applicant]
Office Action for Chinese Application No. 202080044821.9 dated Mar. 21, 2024, 16 pages. [cited by applicant]
Office Action for European Application No. 20818136.2 dated May 23, 2023, 7 pages. [cited by applicant]
Office Action for Korean Application No. 10-2021-7035516 dated Jul. 5, 2024, 11 pages. [cited by applicant]
Outgoing—ISA/210—International Search Report Mailed on Aug. 28, 2020 for WO Application No. PCT/KR20/007221, 5 page(s). [cited by applicant]
Outgoing Written Opinion of the ISA Mailed on Aug. 28, 2020 for WO Application No. PCT/KR20/007221, 7 page(s). [cited by applicant]
Summons to Attend Oral Proceedings for European Application No. 20818136.2 dated Nov. 24, 2023, 12 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2025-103548 dated Mar. 3, 2026, 7 pages. [cited by applicant]