IP Library Granted Patent US 12,273,564
Granted Patent B2
US 12,273,564 · App. 17/234,688 · Granted Apr 8, 2025

Method and device for processing video signal by using subblock-based motion compensation

Inventors: Geonjung Ko (Seoul, KR); Dongcheol Kim (Suwon-Si, KR); Juhyung Son (Uiwang-Si, KR); Jaehong Jung (Seoul, KR); Jinsam Kwak (Anyang-Si, KR)
Assignee: WILUS INSTITUTE OF STANDARDS AND TECHNOLOGY INC.
H04N19/70H04N19/159H04N19/176H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,273,564
App. No.
17/234,688
Granted
Apr 8, 2025
Kind
B2
Abstract

A video signal processing method may comprise the steps of: acquiring a first syntax element indicating whether a subblock-based merging mode is applied to a current block; acquiring a second syntax element indicating whether a merging mode using motion vector difference is applied to the current block, when the first syntax element indicates that the subblock-based merging mode is not applied to the current block; and acquiring a third syntax element indicating a candidate to be used in inter-prediction for the current block, among candidates contained in a mergence candidate list for the current block, when the second syntax element indicates that the merging mode using motion vector difference is applied to the current block.

Claims (27)

1. A method of processing a video signal, the method comprising:

acquiring a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, acquiring a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, acquiring a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, acquiring information related to the motion vector difference, inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without parsing the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

2. The method of claim 1 , wherein the third syntax element indicates the candidate used for the inter prediction of the current block among a first candidate and a second candidate of the merge candidate list of the current block.

3. The method of claim 2 , wherein the third syntax element is parsed from a bitstream when a maximum number of merge candidates is larger than 1, and it is inferred that the third syntax element is 0 when the maximum number of merge candidates is not larger than 1.

4. The method of claim 1 , wherein the acquiring the information related to the motion vector difference comprises:

acquiring a fourth syntax element indicating a distance of the motion vector difference; and

acquiring a fifth syntax element indicating a direction of the motion vector difference.

5. The method of claim 1 , further comprising acquiring a sixth syntax element indicating a candidate used for the inter prediction of the current block among candidates included in a subblock merge candidate list of the current block when the first syntax element indicates that the subblock-based merge mode is applied to the current block, wherein the motion vector of the current block is induced in units of at least one subblock included in the current block, based on a motion vector of the candidate indicated by the sixth syntax element when the first syntax element indicates that the subblock-based merge mode is applied to the current block.

6. The method of claim 5 , wherein, when the first syntax element indicates that the subblock-based merge mode is applied to the current block, it is inferred that the second syntax element is 0.

7. The method of claim 5 , wherein the sixth syntax element is parsed from a bitstream when a maximum number of subblock merge candidates is larger than 1, and it is inferred that the sixth syntax element is 0 when the maximum number of subblock merge candidates is not larger than 1.

8. A method of processing a video signal, the method comprising:

encoding a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, encoding a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, encoding a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, and encoding information related to the motion vector difference for inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without encoding the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

9. A non-transitory computer-readable medium storing a bitstream, the bitstream being decoded by a decoding method,

wherein the decoding method, comprising:

acquiring a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, acquiring a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, acquiring a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, acquiring information related to the motion vector difference, inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without parsing the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2022
From: HUMAX CO., LTD.
To: WILUS INSTITUTE OF STANDARDS AND TECHNOLOGY INC.
Reel/Frame 060456/0196 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 19, 2021
From: KO, GEONJUNG; KIM, DONGCHEOL; SON, JUHYUNG; JUNG, JAEHONG; KWAK, JINSAM
To: WILUS INSTITUTE OF STANDARDS AND TECHNOLOGY INC.; HUMAX CO., LTD.
Reel/Frame 055964/0099 →
Priority Claims (4)
KR 10-2018-0126782 · Oct 23, 2018 · national
KR 10-2018-0137704 · Nov 9, 2018 · national
KR 10-2019-0000479 · Jan 2, 2019 · national
KR 10-2019-0016537 · Feb 13, 2019 · national
Continuity (2)
Continuation PCTKR2019014016 · Oct 23, 2019
Related Publication 20210243476A1 · Aug 5, 2021
References Cited (72)
US 10869050B2 · Li et al. · 2020 [cited by applicant]
US 11172204B2 · Jeong · 2021 [cited by examiner]
US 20120257678A1 · Zhou et al. · 2012 [cited by applicant]
US 20120328209A1 · Sasai et al. · 2012 [cited by applicant]
US 20130070855A1 · Zheng et al. · 2013 [cited by applicant]
US 20130343455A1 · Yamamoto · 2013 [cited by examiner]
US 20170142418A1 · Li et al. · 2017 [cited by applicant]
US 20180220149A1 · Son et al. · 2018 [cited by applicant]
US 20180310017A1 · Chen · 2018 [cited by examiner]
US 20200154124A1 · Lee · 2020 [cited by examiner]
US 20200374528A1 · Huang et al. · 2020 [cited by applicant]
CN 107113424A · 2017 [cited by applicant]
CN 107925758A · 2018 [cited by applicant]
CN 108432250A · 2018 [cited by applicant]
KR 1020130050407A · 2013 [cited by applicant]
KR 1020180000303A · 2018 [cited by applicant]
KR 1020180007345A · 2018 [cited by applicant]
KR 20200084300A · 2020 [cited by applicant]
TW 201817238A · 2018 [cited by applicant]
WO 2017039117A1 · 2017 [cited by applicant]
WO 2017171107A1 · 2017 [cited by applicant]
WO 2018066927A1 · 2018 [cited by applicant]
WO 2020227678A1 · 2020 [cited by applicant]
International Search Report of The International Searching Authority_PCT/KR2019/014016_Feb. 11, 2020. [cited by applicant]
Written Opinion of The International Searching Authoriity_PCT/KR2019/014016_ Feb. 11, 2020. [cited by applicant]
Han. Yu et al. CE4.2.I6: Sub-block merge candidates in BMS and JEM. Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and [SO/IEC JTC 1/SC 29/WG II. JVET-K0338. 11th Meeting: Ljubljana, SI. pp. 1-5, Jul. 18, 2018 … [cited by applicant]
Yu, Ruoyang et al. Non-CE4: On maximum number of subblock merge candidates. Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11. JVET-00220. 5th Meeting. Gothenburg, SE. pp. 1-3, Jul. 12, 2… [cited by applicant]
Office Action from Intellectual Property India for Application No. 202127019359 Dated Feb. 17, 2022. [cited by applicant]
Office Action from Japanese Intellectual Property Office for Japanese Patent Application No. 2021-521968 Dated Apr. 20, 2022. [cited by applicant]
Y. Chen, et al., Description of SDR, HDR and 360° video coding technology proposal considering mobile application scenario by Samsung, Huawei, GoPro, and HiSilicon, Joint Video Exploration Team (JVET) of ITU-TSG 16 WP 3… [cited by applicant]
JVET-J0024 (PPT)Description of SDR, HDR and 360° video coding technology proposal by Samsung, Huawei, GoPro, and HiSilicon—mobile application scenario Apr. 11, 2018. [cited by applicant]
JVET-J0024 and JVET-J0025 (PPT) Description of 360° video coding technology proposal by Samsung, Huawei, GoPro, and HiSilicon Apr. 13, 2018. [cited by applicant]
JVET-J0024_03_360_Set1.xlsm Reference: HM16.16 Reference2:JEM7.0 Tested:P40. [cited by applicant]
JVET-J0024_02_HDR_Set1.xlsm Reference: HM16.16 Reference2:JEM7.0 Tested:JVET-J0024_Set1 Note: the final version of the file including the anchors data will be attached to a revision of the document JVET-H1002. [cited by applicant]
JVET-J0024_01_SDR_Set1.xlsm Reference: HM16.16 Reference2:JEM7.0 Tested:JVET-J0024_Set1 Note: BD-rate and BD-PSNR is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-J0024_01_SDR_optimize.xlsm Reference: HM16.16 Reference2:JEM7.0 Tested:JVET-J0024_Set2 Note: BD-rate and BD-PSNR is computed using piece-wise cubic interpolation. [cited by applicant]
Seungsoo Jeong, et al., CE4 Ultimate motion vector expression in J0024 (Test 4.2.9), Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29/WG 11, JVET-K0115-v4, Jul. 12, 2018. [cited by applicant]
JVET-K0115_CE4.2.9.a_tooloff_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.a_toolon_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b_tooloff_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b_toolon_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b2_tooloff_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b2_toolon_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b3_tooloff_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-K0115_CE4.2.9.b3_toolon_tester.xlsm Reference: VTM-1.0 Tested:Test 2nd Reference:BMS-1.0 Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
Tomonori Hashimoto Eiichi Sasaki Tomohiro Ikai, Non-CE4: Enhanced ultimate motion vector expression, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-L0355, Sep. 25, 2018. [cited by applicant]
JVET-L0355.xlsm Reference: VTM-2.0.1 Tested:vtm201_eumve Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-L0355_8dir.xlsm Reference: VTM-2.0.1_Sharp Tested:vtm201_umve Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
JVET-L0355_list.xlsm Reference: VTM-2.0.1_Sharp Tested:vtm201_umve Note: BD-rate is computed using piece-wise cubic interpolation. [cited by applicant]
The extended European search report for EP Application No. 198757569. Dated Jun. 20, 2022. [cited by applicant]
Yang (Huawei) H: “BoG report on CE4 related contributions”, 12. JVET Meeting; Oct. 3, 2018-Oct. 12, 2018; Macao; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-L0691, Oct. 8, 2018… [cited by applicant]
Yang (Huawei) H et al: “Description of Core Experiment 4 (CE4) : Inter prediction and motion vectior coding”, 11. JVET Meeting; Jul. 11, 2018-Jul. 18, 2018; Ljubljana; (The Joint Video Exploartaion Team of ISO/IEC JTC1/… [cited by applicant]
Ko (Wilusgroup) G et al: “Non-CE4: Modification of merge data syntax”, 13. JVET Meeting; Jan. 9, 2019-Jan. 18, 2019; Marrakech; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-M035… [cited by applicant]
“CE4 Ultimate motion vector expression (Test 4.5.4)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L0054. [cited by applicant]
“Proposed WD for CE4 Ultimate motion vector expression (Test 4.5.4)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L0054_WD. [cited by applicant]
JVET-L0054_CE_4_5_4_a_tester.xlsm. Reference: VTM-2.0.1 Tested: VTM-2.0.1. [cited by applicant]
JVET-L0054_CE_4_5_4_b_tester.xlsm. Reference: VTM-2.0.1 Tested: VTM-2.0.1. [cited by applicant]
JVET-L0054_CE_4_5_4_c_tester.xlsm. Reference: VTM-2.0.1 Tested: VTM-2.0.1. [cited by applicant]
JVET-L0054_CE_4_5_4_d_tester.xlsm. Reference: VTM-2.0.1 Tested: VTM-2.0.1. [cited by applicant]
JVET-L0054_CE_4_5_4_e_tester.xlsm. Reference: VTM-2.0.1 Tested: VTM-2.0.1. [cited by applicant]
“CE4-related: Combination of affine mode clean up and line buffer reduction” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L06… [cited by applicant]
“Versatile Video Coding (Draft 2)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018. Document: JVET-K1001-v7. [cited by applicant]
“Combination of affine mode clean up (L0047 method 1) and line buffer reduction (L0045)” PPT. [cited by applicant]
JVET-L0694.xlsm. Reference: VTM-2.0.1+adopted proposal. Tested: Test. [cited by applicant]
Written Decision on Registration. KR Application No. 10-2021-7012537. Mailed on Mar. 31, 2023. [cited by applicant]
Office Action on EP Application No. 19 875 756.9-1208. Mailed Mar. 28, 2023. [cited by applicant]
Notice of Allowance for CN Patent Application No. 201980069820.7 from China National Intellectual Property Administration. Date of Issue: Jul. 6, 2023. [cited by applicant]
(joint Video Experts Team (JVET] of ITU-T SG16WP3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, Sep. 25, 2018, Huanbang Chen,CE4: Symmetrical MVD mode (Test 4.5.1). [cited by applicant]
Notice of Allowance Action from Japanese Intellectual Property Office for Japanese Patent Application No. 2022-212171 Issue Date: Nov. 6, 2023. Title: A video signal processing method and apparatus using the compensatio… [cited by applicant]
Title: CE4-1.3: Modification of merge data syntax Document: JVET-N0237 Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019. [cited by applicant]
Office Action from European Patent Office for Application No. 19875756.9-1208. Issue Date: Nov. 15, 2023. Applicant: WILUS Institute of Standards and Technology Inc. [cited by applicant]
Office Action from Korean Intellectual Property Office for Application No. 10-2023-7021844 Dated Jan. 19, 2024. Applicant: WILUS Institute of Standards and Technology Inc. Title: Method and Device for Processing Video S… [cited by applicant]