IP Library Granted Patent US 12,695,882
Granted Patent B2
US 12,695,882 · App. 18/885,435 · Granted Jul 28, 2026

Method for processing image on basis of inter-prediction mode and device therefor

Inventor: Hyeongmoon Jang (Seoul, KR)
Assignee: NOKIA TECHNOLOGIES OY
H04N19/159H04N19/105H04N19/137H04N19/176H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,695,882
App. No.
18/885,435
Filed
Sep 13, 2024
Granted
Jul 28, 2026
Kind
B2
Examiner
HABIB, IRFAN
Art Unit
2485
USPC
375/240.16
Abstract

Disclosed herein are a method for decoding a video signal and a device therefor. Specifically, a method for decoding an image based on an inter prediction mode may include: if a merge mode is applied to a current block, generating a merge candidate list by using a spatial neighboring block and a temporal neighboring block of the current block; obtaining a merge index indicating a candidate to be used for an inter prediction of the current block in the merge candidate list; deriving a motion vector of each of subblocks included in the current block based on a motion vector of the candidate used for the inter prediction of the current block; and generating a prediction sample of the current block by using the motion vector of each of subblocks.

Claims (42)

1 . An apparatus comprising:

a memory configured to store an image; and

a processor coupled with the memory,

wherein the processor is configured to:

generate a merge candidate list including at least one of a spatial neighboring block and a temporal neighboring block of a current block, and including candidates equal to or less than a maximum number of merge candidates, based on that a merge mode is applied to the current block;

obtain a merge index indicating a candidate to be used for an inter prediction of the current block in the merge candidate list;

derive motion vectors of each subblocks included in the current block based on motion vectors of the candidate indicated by the merge index;

generate a prediction block of the current block based on the motion vectors of each subblocks;

perform inverse-transform on transformed coefficients of the current block to generate a residual block of the current block; and

generate a reconstructed block of the current block based on the prediction block and the residual block,

wherein the processor is further configured to:

based on that a width and a height of the current block are respectively greater than or equal to a pre-defined minimum size, add the temporal neighboring block which is located in a collocated picture of the current block and specified by a motion vector of the spatial neighboring block to the merge candidate list,

wherein based on that the width or the height of the current block is smaller than the pre-defined minimum size, the temporal neighboring block is not included in the merge candidate list, and

wherein based on that the merge index indicates the temporal neighboring block as the candidate, the motion vectors of the each subblocks included in the current block are derived based on motion vectors of subblocks included in the temporal neighboring block.

2 . The apparatus of claim 1 , wherein the temporal neighboring block is specified by a motion vector of a first spatial neighboring block of the merge candidate list, the first spatial neighboring block being determined based on a pre-defined order among two or more spatial neighboring blocks of the current block.

3 . The apparatus of claim 1 , wherein the pre-defined minimum size is set to 8.

4 . An apparatus comprising:

a memory configured to store an image; and

a processor coupled with the memory,

wherein the processor is configured to:

generate a merge candidate list including at least one of a spatial neighboring block and a temporal neighboring block of a current block, and including candidates equal to or less than a maximum number of merge candidates, based on that a merge mode is applied to the current block;

derive motion vectors of each subblocks included in the current block based on motion vectors of a candidate included in the merge candidate list;

generate a prediction block of the current block based on the motion vectors of each subblocks;

generate a residual block of the current block based on the prediction block;

generate a merge index indicating the candidate in the merge candidate list; and

perform transform on the residual block,

wherein the processor is further configured to:

based on that a width and a height of the current block being respectively greater than or equal to a pre-defined minimum size, add the temporal neighboring block which is located in a collocated picture of the current block and specified by a motion vector of the spatial neighboring block to the merge candidate list,

wherein based on that the width or the height of the current block is smaller than the pre-defined minimum size, the temporal neighboring block is not included in the merge candidate list, and

wherein based on the motion vector of the temporal neighboring block in the merge candidate list being used for deriving the motion vectors of the each subblocks of the current block, the motion vectors of the each subblocks of the current block are derived based on motion vectors of subblocks divided from the temporal neighboring block.

5 . The apparatus of claim 4 , wherein the temporal neighboring block is specified by a motion vector of a first spatial neighboring block of the merge candidate list, the first spatial neighboring block being determined based on a pre-defined order among two or more spatial neighboring blocks of the current block.

6 . The method of claim 4 , wherein the pre-defined minimum size is set to 8.

7 . An apparatus comprising:

a processor configured to generate a bitstream for an image; and

a transmitter configured to transmit data comprising the bitstream,

wherein the processor is configured to:

generate a bitstream for the image, the bitstream is generated by generating a merge candidate list including at least one of a spatial neighboring block and a temporal neighboring block of a current block, and including candidates equal to or less than a maximum number of merge candidates, based on that a merge mode is applied to the current block; deriving motion vectors of each subblocks included in the current block based on motion vectors of a candidate included in the merge candidate list; generating a prediction block of the current block based on the motion vectors of each subblocks; generating a residual block of the current block based on the prediction block; generating a merge index indicating the candidate in the merge candidate list, and performing transform on the residual block, and

transmit data comprising the bitstream,

wherein the processor is configured to:

based on that a width and a height of the current block being respectively greater than or equal to a pre-defined minimum size, add the temporal neighboring block which is located in a collocated picture of the current block and specified by a motion vector of the spatial neighboring block to the merge candidate list,

wherein based on that the width or the height of the current block is smaller than the pre-defined minimum size, the temporal neighboring block is not included in the merge candidate list, and

wherein based on the motion vector of the temporal neighboring block in the merge candidate list being used for deriving the motion vectors of the each subblocks of the current block, the motion vectors of the each subblocks of the current block are derived based on motion vectors of subblocks divided from the temporal neighboring block.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 9, 2025
From: LG ELECTRONICS INC.
To: NOKIA TECHNOLOGIES OY
Reel/Frame 072859/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2024
From: JANG, HYEONGMOON
To: LG ELECTRONICS INC.
Reel/Frame 068588/0222 →
Priority Claims (1)
KR 10-2018-0074069 · Jun 27, 2018 · national
Continuity (5)
Continuation 18134808 · Apr 14, 2023
Continuation 17360734 · Jun 28, 2021
Continuation 16943908 · Jul 30, 2020
Continuation PCTKR2019007837 · Jun 27, 2019
Related Publication 20250008115A1 · Jan 2, 2025
References Cited (92)
US 10368083B2 · Chien · 2019 [cited by examiner]
US 10382738B2 · Kim · 2019 [cited by examiner]
US 10397603B1 · Li et al. · 2019 [cited by applicant]
US 10469869B1 · Li · 2019 [cited by examiner]
US 10674175B2 · Park · 2020 [cited by examiner]
US 10820007B2 · Park · 2020 [cited by examiner]
US 10841576B2 · Lee · 2020 [cited by examiner]
US 20120236942A1 · Lin et al. · 2012 [cited by applicant]
US 20130294512A1 · Song et al. · 2013 [cited by applicant]
US 20140294086A1 · Oh et al. · 2014 [cited by applicant]
US 20140314143A1 · Lee et al. · 2014 [cited by applicant]
US 20150281708A1 · Chuang et al. · 2015 [cited by applicant]
US 20160219278A1 · Chen et al. · 2016 [cited by applicant]
US 20160295240A1 · Kim · 2016 [cited by examiner]
US 20160330475A1 · Zhou · 2016 [cited by examiner]
US 20160344903A1 · Zhou · 2016 [cited by examiner]
US 20160366435A1 · Chien et al. · 2016 [cited by applicant]
US 20160381382A1 · Oh et al. · 2016 [cited by applicant]
US 20170019680A1 · Lee · 2017 [cited by examiner]
US 20170188028A1 · Park et al. · 2017 [cited by applicant]
US 20170332099A1 · Lee et al. · 2017 [cited by applicant]
US 20180070090A1 · Lee et al. · 2018 [cited by applicant]
US 20180070100A1 · Chen et al. · 2018 [cited by applicant]
US 20180192053A1 · Zhou · 2018 [cited by examiner]
US 20190200040A1 · Lim et al. · 2019 [cited by applicant]
US 20190313112A1 · Han et al. · 2019 [cited by applicant]
US 20190320182A1 · Oh et al. · 2019 [cited by applicant]
US 20200045307A1 · Jang · 2020 [cited by examiner]
US 20200053380A1 · Chen · 2020 [cited by examiner]
US 20200221077A1 · Park · 2020 [cited by examiner]
US 20200228796A1 · Seregin · 2020 [cited by examiner]
US 20200275110A1 · Galpin · 2020 [cited by examiner]
US 20200280735A1 · Lim · 2020 [cited by examiner]
CN 107105286A · 2017 [cited by applicant]
CN 107211156A · 2017 [cited by applicant]
CN 107404652A · 2017 [cited by applicant]
CN 107465921A · 2017 [cited by applicant]
CN 107635137A · 2018 [cited by applicant]
CN 110460857A · 2019 [cited by applicant]
JP 2013118627A · 2013 [cited by applicant]
JP 2017163584A · 2017 [cited by applicant]
JP 7605950B2 · 2024 [cited by applicant]
KR 1020130008481A · 2013 [cited by applicant]
KR 1020130063044A · 2013 [cited by applicant]
KR 1020130085376A · 2013 [cited by applicant]
KR 1020150105264A · 2015 [cited by applicant]
KR 1020180007345A · 2018 [cited by applicant]
KR 1020180038371A · 2018 [cited by applicant]
WO 2017039117A1 · 2017 [cited by applicant]
WO 2017197126A1 · 2017 [cited by applicant]
WO 2018012886A1 · 2018 [cited by applicant]
WO 2018026118A1 · 2018 [cited by applicant]
WO 2018065296A1 · 2018 [cited by applicant]
WO 2018066874A1 · 2018 [cited by applicant]
WO 2018066927A1 · 2018 [cited by applicant]
WO 2018097692A2 · 2018 [cited by applicant]
WO 2018101700A1 · 2018 [cited by applicant]
Decision of Refusal for Japanese Application No. 2024-217267 dated Nov. 11, 2025, 6 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2020-547121 dated Jul. 12, 2022, 6 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2022-143392 dated Oct. 31, 2023, 5 pages. [cited by applicant]
Decision to Grant for Japanese Application No. 2023-201648 dated Nov. 12, 2024, 5 pages. [cited by applicant]
Decision to Grant for Korean Application No. 10-2022-70023732 dated Jun. 5, 2023, 4 pages. [cited by applicant]
Extended European Search Report for European Application No. 19824904.7 dated Feb. 2, 2021, 10 pages. [cited by applicant]
H. Jang et al., “[CE4-2.6 related] Simplified ATMVP with fixed sub-block size”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 10-18, 2018, JVET-K0080-v1. [cited by applicant]
IPEA/409—International Preliminary Report on Patentability Mailed on Jan. 7, 2021 for WO Application No. PCT/KR19/007837, 12 page(s). [cited by applicant]
J. Chen et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7)”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 13-21, 2017, JVET-G1001-v1. [cited by applicant]
J. Chen et al., “Coding tools investigation for next generation video coding based on HEVC” Proceedings of SPIE, IEEE, US, vol. 9599, Sep. 22, 2015, pp. 95991B-1-95991B-9, XP060060836. [cited by applicant]
Non-Final Rejection Mailed on Dec. 7, 2023 for U.S. Appl. No. 18/134,808, 11 page(s). [cited by applicant]
Non-Final Rejection Mailed on Nov. 23, 2020 for U.S. Appl. No. 16/943,908, 8 page(s). [cited by applicant]
Non-Final Rejection Mailed on Sep. 2, 2022 for U.S. Appl. No. 17/360,734, 9 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Jan. 11, 2023 for U.S. Appl. No. 17/360,734, 5 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Mar. 26, 2021 for U.S. Appl. No. 16/943,908, 5 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on May 21, 2024 for U.S. Appl. No. 18/134,808, 5 page(s). [cited by applicant]
Notice of Allowance for Chinese Application No. 201980024218.1 dated May 29, 2023, 10 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2020-547121 dated Mar. 8, 2022, 6 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2020547121 dated Oct. 12, 2021, 8 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2022-143392 dated Jul. 11, 2023, 6 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2023-201648 dated Jul. 16, 2024, 6 pages. [cited by applicant]
Notice of Reasons for Refusal for Japanese Application No. 2024-217267 dated Jul. 15, 2025, 6 pages. [cited by applicant]
Office Action for Korean Application No. 10-2020-7016708 dated Oct. 29, 2021, 7 pages. [cited by applicant]
Office Action for Korean Application No. 10-2022-7023732 dated Feb. 23, 2023, 6 pages. [cited by applicant]
Office Action for Korean Application No. 10-2023-7030124 dated Apr. 16, 2024, 5 pages. [cited by applicant]
Outgoing—ISA/210—International Search Report Mailed on Oct. 23, 2019 for WO Application No. PCT/KR19/007837, 5 page(s). [cited by applicant]
Outgoing Written Opinion of the ISA Mailed on Oct. 23, 2019 for WO Application No. PCT/KR19/007837, 9 page(s). [cited by applicant]
Written Decision on Registration for Korean Application No. 10-2020-7016708 dated Apr. 12, 2022, 3 pages. [cited by applicant]
Written Decision on Registration for Korean Application No. 10-2023-7030124 dated Dec. 3, 2024, 5 pages. [cited by applicant]
X. Xiu et al., “Description of SDR, HDR, and 360° video coding technology proposal by InterDigital Communications and Dolby Laboratories”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 1… [cited by applicant]
“Summary of ATMVP and STMVP”, CSDN Blog, (Mar. 16, 2018), 20 pages. [cited by applicant]
Office Action for Chinese Application No. 202311013940.X dated Mar. 27, 2026, 11 pages. [cited by applicant]
Office Action for Chinese Application No. 202311015983.1 dated Apr. 1, 2026, 11 pages. [cited by applicant]
Sullivan et al., “Meeting Report of the 10th meeting of the Joint Video Experts Team (JVET), San Diego, US, Apr. 10-20, 2018”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meet… [cited by applicant]
Sun et al., “Fast Inter Mode Decision Algorithm Based on Block Spatio-Temporal Correlation in H.264”, J-Global, vol. 26, Issue 3, (2009), 4 pages. [cited by applicant]