IP Library › Granted Patent US 12,273,566
Granted Patent B2
US 12,273,566 · App. 18/360,645 · Granted Apr 8, 2025

Method and device for processing video signal by using subblock-based motion compensation

Inventors: Geonjung Ko (Seoul, KR); Dongcheol Kim (Suwon-Si, KR); Juhyung Son (Uiwang-Si, KR); Jaehong Jung (Seoul, KR); Jinsam Kwak (Anyang-Si, KR)
Assignee: WILUS INSTITUTE OF STANDARDS AND TECHNOLOGY INC.
H04N19/70H04N19/159H04N19/176H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,273,566
App. No.
18/360,645
Granted
Apr 8, 2025
Kind
B2
Abstract

A video signal processing method may comprise the steps of: acquiring a first syntax element indicating whether a subblock-based merging mode is applied to a current block; acquiring a second syntax element indicating whether a merging mode using motion vector difference is applied to the current block, when the first syntax element indicates that the subblock-based merging mode is not applied to the current block; and acquiring a third syntax element indicating a candidate to be used in inter-prediction for the current block, among candidates contained in a mergence candidate list for the current block, when the second syntax element indicates that the merging mode using motion vector difference is applied to the current block.

Claims (23)

1. A method of processing a video signal, the method comprising:

acquiring a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, acquiring a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, acquiring a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, acquiring a fourth syntax element indicating a distance of the motion vector difference, acquiring a fifth syntax element indicating a direction of the motion vector difference, and inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates the candidate used for the inter prediction of the current block among a first candidate and a second candidate of the merge candidate list of the current block;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without parsing the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

2. The method of claim 1 , wherein the third syntax element is parsed from a bitstream when a maximum number of merge candidates is larger than 1, and it is inferred that the third syntax element is 0 when the maximum number of merge candidates is not larger than 1.

3. The method of claim 1 , further comprising acquiring a sixth syntax element indicating a candidate used for the inter prediction of the current block among candidates included in a subblock merge candidate list of the current block when the first syntax element indicates that the subblock-based merge mode is applied to the current block, wherein the motion vector of the current block is induced in units of at least one subblock included in the current block, based on a motion vector of the candidate indicated by the sixth syntax element when the first syntax element indicates that the subblock-based merge mode is applied to the current block.

4. The method of claim 3 , wherein, when the first syntax element indicates that the subblock-based merge mode is applied to the current block, it is inferred that the second syntax element is 0.

5. The method of claim 3 , wherein the sixth syntax element is parsed from a bitstream when a maximum number of subblock merge candidates is larger than 1, and it is inferred that the sixth syntax element is 0 when the maximum number of subblock merge candidates is not larger than 1.

6. A method of processing a video signal, the method comprising:

encoding a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, encoding a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, encoding a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, and encoding information related to the motion vector difference for inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates the candidate used for the inter prediction of the current block among a first candidate and a second candidate of the merge candidate list of the current block, wherein encoding information related to the motion vector difference includes encoding a fourth syntax element indicating a distance of the motion vector difference and a fifth syntax element indicating a direction of the motion vector difference;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without encoding the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

7. A non-transitory computer-readable medium storing a bitstream, the bitstream being decoded by a decoding method,

wherein the decoding method, comprising:

acquiring a first syntax element indicating whether a subblock-based merge mode is applied to a current block;

when the first syntax element indicates that the subblock-based merge mode is not applied to the current block, acquiring a second syntax element indicating whether a merge mode using a motion vector difference is applied to the current block;

when the second syntax element indicates that the merge mode using the motion vector difference is applied to the current block, acquiring a third syntax element indicating a candidate used for inter prediction of the current block among candidates included in a merge candidate list of the current block, acquiring a fourth syntax element indicating a distance of the motion vector difference, acquiring a fifth syntax element indicating a direction of the motion vector difference, and inducing a motion vector of the current block by adding the motion vector difference to a motion vector of the candidate indicated by the third syntax element, wherein the third syntax element indicates the candidate used for the inter prediction of the current block among a first candidate and a second candidate of the merge candidate list of the current block;

when the first syntax element indicates that the subblock-based merge mode is applied to the current block, inducing the motion vector of the current block without parsing the second syntax element; and

generating a predicted block of the current block using the motion vector of the current block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2023
From: KO, GEONJUNG; KIM, DONGCHEOL; SON, JUHYUNG; JUNG, JAEHONG; KWAK, JINSAM
To: WILUS INSTITUTE OF STANDARDS AND TECHNOLOGY INC.
Reel/Frame 064410/0100 →
Priority Claims (4)
KR 10-2018-0126782 · Oct 23, 2018 · national
KR 10-2018-0137704 · Nov 9, 2018 · national
KR 10-2019-0000479 · Jan 2, 2019 · national
KR 10-2019-0016537 · Feb 13, 2019 · national
Continuity (3)
Continuation 17234688 · Apr 19, 2021
Continuation PCTKR2019014016 · Oct 23, 2019
Related Publication 20240040156A1 · Feb 1, 2024
References Cited (55)
US 10869050B2 · Li et al. · 2020 [cited by applicant]
US 11172204B2 · Jeong · 2021 [cited by applicant]
US 20120257678A1 · Zhou et al. · 2012 [cited by applicant]
US 20120328209A1 · Sasai et al. · 2012 [cited by applicant]
US 20130070855A1 · Zheng et al. · 2013 [cited by applicant]
US 20130343455A1 · Yamamoto · 2013 [cited by applicant]
US 20170142418A1 · Li et al. · 2017 [cited by applicant]
US 20180220149A1 · Son et al. · 2018 [cited by applicant]
US 20180310017A1 · Chen · 2018 [cited by applicant]
US 20200154124A1 · Lee · 2020 [cited by applicant]
US 20200374528A1 · Huang et al. · 2020 [cited by applicant]
CN 107113424A · 2017 [cited by applicant]
CN 107925758A · 2018 [cited by applicant]
CN 108432250A · 2018 [cited by applicant]
KR 1020130050407A · 2013 [cited by applicant]
KR 1020180000303A · 2018 [cited by applicant]
KR 1020180007345A · 2018 [cited by applicant]
KR 20200084300A · 2020 [cited by applicant]
TW 201817238A · 2018 [cited by applicant]
WO 2017039117A1 · 2017 [cited by applicant]
WO 2017171107A1 · 2017 [cited by applicant]
WO 2018066927A1 · 2018 [cited by applicant]
WO 2020227678A1 · 2020 [cited by applicant]
International Search Report of the International Searching Authority_PCT/KR2019/014016_Feb. 11, 2020. [cited by applicant]
Written Opinion of the International Searching Authoriity_PCT/KR2019/014016_ Feb. 11, 2020. [cited by applicant]
Han. Yu et al. CE4.2.16: Sub-block merge candidates in BMS and JEM. Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and [SO/IEC JTC 1/SC 29/WG I I. JVET-K0338. 11th Meeting: Ljubljana, SI. pp. 1-5, Jul. 18, 2018… [cited by applicant]
Yu, Ruoyang et al. Non-CE4: On maximum number of subblock merge candidates. Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11. JVET-00220. 5th Meeting. Gothenburg, SE. pp. 1-3, Jul. 12, 2… [cited by applicant]
Office Action from Intellectual Property India for Application No. 202127019359 Dated Feb. 17, 2022. [cited by applicant]
Office Action from Japanese Intellectual Property Office for Japanese Patent Application No. 2021-521968 Dated Apr. 20, 2022. [cited by applicant]
Y. Chen, et al., Description of SDR, HDR and 360° video coding technology proposal considering mobile application scenario by Samsung, Huawei, GoPro, and HiSilicon, Joint Video Exploration Team (JVET) of ITU-TSG 16 WP 3… [cited by applicant]
JVET-J0024 (PPT)Description of SDR, HDR and 360° video coding technology proposal by Samsung, Huawei, GoPro, and HiSilicon—mobile application scenario Apr. 11, 2018. [cited by applicant]
JVET-J0024 and JVET-J0025 (PPT) Description of 360° video coding technology proposal by Samsung, Huawei, GoPro, and HiSilicon Apr. 13, 2018. [cited by applicant]
JVET-J0024_01_SDR_optimize.xlsm Reference: HM16.16 Reference2:JEM7.0 Tested:JVET-J0024_Set2 Note: BD-rate and BD-PSNR is computed using piece-wise cubic interpolation. [cited by applicant]
Seungsoo Jeong, et al.,CE4 Ultimate motion vector expression in J0024 (Test 4.2.9), Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29/WG 11, JVET-K0115-v4, Jul. 12, 2018. [cited by applicant]
Tomonori Hashimoto Eiichi Sasaki Tomohiro Ikai,Non-CE4: Enhanced ultimate motion vector expression, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-L0355, Sep. 25, 2018. [cited by applicant]
The extended European search report for EP Application No. 198757569. Dated Jun. 20, 2022. [cited by applicant]
Yang (Huawei) H: “BoG report on CE4 related contributions”, 12. JVET Meeting; Oct. 3, 2018-Oct. 12, 2018; Macao; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-L0691, Oct. 8, 2018… [cited by applicant]
Yang (Huawei) H et al: “Description of Core Experiment 4 (CE4) : Inter prediction and motion vectior coding”, 11. JVET Meeting; Jul. 11, 2018-Jul. 18, 2018; Ljubljana; (The Joint Video Exploartaion Team of ISO/IEC JTC1/… [cited by applicant]
Ko (Wilusgroup) G et al: “Non-CE4: Modification of merge data syntax”, 13. JVET Meeting; Jan. 9, 2019-Jan. 18, 2019; Marrakech; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-M035… [cited by applicant]
“CE4 Ultimate motion vector expression (Test 4.5.4)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L0054. [cited by applicant]
“Proposed WD for CE4 Ultimate motion vector expression (Test 4.5.4)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L0054_WD. [cited by applicant]
“CE4-related: Combination of affine mode clean up and line buffer reduction” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018. Document: JVET-L06… [cited by applicant]
“Versatile Video Coding (Draft 2)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/ WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018. Document: JVET-K1001-v7. [cited by applicant]
Written Decision on Registration. KR Application No. 10-2021-7012537. Mailed on Mar. 31, 2023. [cited by applicant]
Office Action on EP Application No. 19 875 756.9-1208. Mailed Mar. 28, 2023. [cited by applicant]
Notice of Allowance for CN Patent Application No. 201980069820.7 from China National Intellectual Property Administration. Date of Issue: Jul. 6, 2023. [cited by applicant]
(Joint Video Experts Team (JVET] of ITU-T SG16WP3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, Sep. 25, 2018, Huanbang Chen, CE4: Symmetrical MVD mode (Test 4.5.1). [cited by applicant]
Non-Final Rejection for U.S. Appl. No. 17/234,688 from USPTO dated Nov. 4, 2022. [cited by applicant]
Notice of Allowance or U.S. Appl. No. 17/234,688 from USPTO dated Apr. 24, 2023. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/234,688 from USPTO dated Feb. 12, 2024. [cited by applicant]
Request for the Submission of an Opinion for KR Application No. 10-2023-7021844 from KIPO. Dated Jan. 9, 2024. [cited by applicant]
Notice of Allowance from Japanese Intellectual Property Office for Japanese Patent Application No. 2022-212171 Dated Oct. 27, 2023. Title: A video signal processing method and apparatus using the compensation of block-b… [cited by applicant]
Seungsoo Jeong, et al. Title: Proposed WD for CE4 Ultimate motion vector expression (Test 4.5.4), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0054_WD, 12th Meeting: Macao, CN… [cited by applicant]
Geonjung Ko, et al. Title: CE4-1.3: Modification of merge data syntax, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-N0237, 14th Meeting: Geneva, CH, Mar. 19-27, 2019. [cited by applicant]
Office Action on EP Application No. 19 875 756.9-1208. Mailed Nov. 15, 2023. [cited by applicant]