IP Library › Granted Patent US 12,309,416
Granted Patent B2
US 12,309,416 · App. 18/443,823 · Granted May 20, 2025

Side motion refinement in video encoding/decoding systems

Inventors: Du Liu (Solna, SE); Rickard Sjöberg (Stockholm, SE); Jacob Ström (Stockholm, SE); Ruoyang Yu (Täby, SE)
Assignee: Telefonaktiebolaget LM Ericsson (publ)
H04N19/513H04N19/577
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,309,416
App. No.
18/443,823
Granted
May 20, 2025
Kind
B2
Abstract

A method for decoding a current block in a current picture of a video bitstream includes decoding, from the video bitstream, a first motion vector for the current block relative to a first reference block of a first reference picture having a first picture order count, and decoding, from the video bitstream, a second motion vector for the current block relative to a second reference block of a second reference picture having a second picture order count. A similarity metric is generated based on a comparison of the first motion vector and the second motion vector. The method further includes determining whether to refine the first motion vector based on the similarity metric, generating a first refined motion vector from the first motion vector, and performing motion compensation to derive a first reference block from the first reference picture using the first refined motion vector.

Claims (24)

1. A method, performed by a decoder, for decoding a current block in a current picture of a video bitstream, the current picture having a current picture order count, the method comprising:

decoding, from the video bitstream, a first motion vector for the current block relative to a first reference block of a first reference picture having a first picture order count;

decoding, from the video bitstream, a second motion vector for the current block relative to a second reference block of a second reference picture having a second picture order count, wherein the first and second motion vectors each comprise a three-dimensional motion vector including an x-component in a plane of the current picture, a y-component in the plane of the current picture and a z-component, wherein the z-component represents a time component;

determining whether to refine the first motion vector based on comparing a difference between the z-components of the first and second motion vectors to a third threshold, and refining the first motion vector in response to the difference between the z-components of the first and second motion vectors being less than the third threshold;

responsive to determining to refine the first motion vector, generating a first refined motion vector from the first motion vector; and

performing motion compensation to derive an updated first reference block from the first reference picture using the first refined motion vector.

2. The method of claim 1 , wherein the z-component of the first motion vector comprises a difference between the current picture order count and the first picture order count, and the z-component of the second motion vector comprises a difference between the current picture order count and the second picture order count; and

further comprising generating the difference between the z-components of the first and second motion vectors based on a difference between an absolute value of the z-component of the first motion vector and an absolute value of the z-component of the second motion vector.

3. The method of claim 1 , wherein determining whether to refine the first motion vector further comprises comparing a difference between the x-components of the first and second motion vectors to a first threshold and determining to refine the first motion vector in response to the difference between the x-components of the first and second motion vectors being less than the first threshold.

4. The method of claim 3 , further comprising generating the difference between the x-components of the first and second motion vectors based on a difference between an absolute value of the x-component of the first motion vector and an absolute value of the x-component of the second motion vector.

5. The method of claim 1 , wherein determining whether to refine the first motion vector further comprises comparing a difference between the y-components of the first and second motion vectors to a second threshold and determining to refine the first motion vector in response to the difference between the y-components of the first and second motion vectors being less than the second threshold.

6. The method of claim 5 , further comprising generating the difference between the y-components of the first and second motion vectors based on a difference between an absolute value of the y-component of the first motion vector and an absolute value of the y-component of the second motion vector.

7. A method, performed by an encoder, for encoding a current block in a current picture of a video bitstream, the current picture having a current picture order count, the method comprising:

generating a first motion vector for the current block relative to a first reference block of a first reference picture having a first picture order count;

generating a second motion vector for the current block relative to a second reference block of a second reference picture having a second picture order count, wherein the first and second motion vectors each comprise a three-dimensional motion vector including an x-component in a plane of the current picture, a y-component in the plane of the current picture and a z-component, wherein the z-component represents a time component;

determining whether to refine the first motion vector based on comparing a difference between the z-components of the first and second motion vectors to a third threshold, and refining the first motion vector in response to the difference between the z-components of the first and second motion vectors being less than the third threshold;

responsive to determining to refine the first motion vector, generating a first refined motion vector from the first motion vector; and

performing motion compensation to derive an updated first reference block from the first reference picture using the first refined motion vector.

8. The method of claim 7 , wherein the z-component of the first motion vector comprises a difference between the current picture order count and the first picture order count, and the z-component of the second motion vector comprises a difference between the current picture order count and the second picture order count; and

further comprising generating the difference between the z-components of the first and second motion vectors comprises generating a difference between an absolute value of the z-component of the first motion vector and an absolute value of the z-component of the second motion vector.

9. The method of claim 7 , wherein determining whether to refine the first motion vector further comprises comparing a difference between the x-components of the first and second motion vectors to a first threshold and determining to refine the first motion vector in response to the difference between the x-components of the first and second motion vectors being less than the first threshold.

10. The method of claim 9 , further comprising generating the difference between the x-components of the first and second motion vectors based on a difference between an absolute value of the x-component of the first motion vector and an absolute value of the x-component of the second motion vector.

11. The method of claim 7 , wherein determining whether to refine the first motion vector further comprises comparing a difference between the y-components of the first and second motion vectors to a second threshold and determining to refine the first motion vector in response to the difference between the y-components of the first and second motion vectors being less than the second threshold.

12. The method of claim 11 , further comprising generating the difference between the y-components of the first and second motion vectors based on a difference between an absolute value of the y-component of the first motion vector and an absolute value of the y-component of the second motion vector.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 16, 2024
From: LIU, DU; SJÖBERG, RICKARD; STRÖM, JACOB; YU, RUOYANG
To: TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
Reel/Frame 066482/0606 →
Continuity (4)
Continuation 17567372 · Jan 3, 2022
Continuation 16639777
Provisional Application 62787587 · Jan 2, 2019
Related Publication 20240259581A1 · Aug 1, 2024
References Cited (41)
US 10375413B2 · Li et al. · 2019 [cited by applicant]
US 20100054338A1 · Suzuki · 2010 [cited by examiner]
US 20130101042A1 · Sugio et al. · 2013 [cited by applicant]
US 20130342644A1 · Rusanovskyy et al. · 2013 [cited by applicant]
US 20140301463A1 · Rusanovskyy et al. · 2014 [cited by applicant]
US 20160073111A1 · Lee et al. · 2016 [cited by applicant]
US 20160269639A1 · Chang · 2016 [cited by examiner]
US 20160286230A1 · Li et al. · 2016 [cited by applicant]
US 20180192071A1 · Chuang et al. · 2018 [cited by applicant]
US 20180262773A1 · Chuang et al. · 2018 [cited by applicant]
US 20180278949A1 · Karczewicz · 2018 [cited by examiner]
US 20180316929A1 · Li et al. · 2018 [cited by applicant]
US 20180359486A1 · Lai et al. · 2018 [cited by applicant]
US 20180376166A1 · Chuang et al. · 2018 [cited by applicant]
US 20190110058A1 · Chien et al. · 2019 [cited by applicant]
CN 10375413A · 2018 [cited by applicant]
CO 14007528A · 2014 [cited by applicant]
EP 3264768A1 · 2018 [cited by applicant]
JP 2020108012A · 2020 [cited by applicant]
WO 2012172113A1 · 2012 [cited by applicant]
WO 2017036414A1 · 2017 [cited by applicant]
WO 2019001737A1 · 2019 [cited by applicant]
WO 2019001785A1 · 2019 [cited by applicant]
WO 2019072371A1 · 2019 [cited by applicant]
WO 2019072373A1 · 2019 [cited by applicant]
WO 2019105536A1 · 2019 [cited by applicant]
Office Action mailed Dec. 29, 2023 for Colombia Patent Application No. NC2021/0010122, 18 pages (includes English translation). [cited by applicant]
Yu, Ru and Liu, D.; “Non-CE9: On motion refinement parameter derivation in BDOF”; Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11; Mar. 19-27, 2019; Geneva, Switzerland; 5 pages. [cited by applicant]
E. Alshina and .A. Alshin, “Bi-directional optical flow,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-C204, Guangzhou, China, Oct. 10-15, 2010. [cited by applicant]
A., Alexander, and E. Alshina, “Bi-directional pptical flow for future video codec.” In Data Compression Conference (DCC), 2016, pp. 83-90. IEEE, 2016. [cited by applicant]
X. Xiu, Y. He, Y. Ye, “CE9-related: Complexity reduction and bit-width control for bi-directional optical flow (BIO)”, Input document to JVET, document No. JVET-L0256, Oct. 3-12, 2018, 15 pages. [cited by applicant]
Examination Report mailed Mar. 10, 2022 for Indian Patent Application No. 202147034075, 6 pages. [cited by applicant]
Extended European Search Report mailed Oct. 25, 2022 for European Patent Application No. 19907770.2, 12 pages. [cited by applicant]
Office Action mailed Oct. 18, 2022 for Japanese Patent Application No. 2021-538229, 5 pages. [cited by applicant]
Alshina et al.; “Known Tools Performance Investigation for Next Generation Video Coding”; ITU—Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Group (VCEG), Document VCEG-AZ05; … [cited by applicant]
Esenlik et al.; “CE9: DMVR with Motion Vector Difference Mirroring (Test 1.6)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-K0216-v3; Jul. 10-18, 2018; 6 pages. [cited by applicant]
Esenlik et al.; “Simplified DMVR for Inclusion in VVC”; Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0670; Oct. 3-12, 2018; 5 pages. [cited by applicant]
Liu et al.; “CE9-Related: Disabling DMVR and BDOF When Underlying Assumptions are False”; Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-N0440; Mar. 19-27, 2019; 2 pages. [cited by applicant]
Liu et al.; “CE9-Related: Motion Vector Refinement in Bi-Directional Optical Flow”; Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0333-v1; Oct. 3-12, 2018; 3 pages. [cited by applicant]
Chinese First Office Action dated Jan. 4, 2024 for Chinese Patent Application No. 201980087686.3, 7 pages. [cited by applicant]
Huawei Technologies Co., Ltd. et al., “Simplified DMVR for inclusion in VVC,” Joint Video Experts Team (JVET) of ITU-T SG 106 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0670, 12th Meeting, Macao, China, Oct. 3-12, 2018, … [cited by applicant]