IP Library Granted Patent US 12,244,791
Granted Patent B2
US 12,244,791 · App. 18/510,196 · Granted Mar 4, 2025

History-based motion vector prediction

Inventors: Xiaoyu Xiu (San Diego, CA); Yuwen He (San Diego, CA); Jiancong Luo (Skillman, NJ)
Assignee: InterDigital VC Holdings, Inc.
H04N19/105H04N19/159H04N19/176H04N19/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,244,791
App. No.
18/510,196
Granted
Mar 4, 2025
Kind
B2
Abstract

Systems, methods, and instrumentalities are disclosed for processing history-based motion vector prediction (HMVP). A video coding device may generate a history-based motion vector prediction (HMVP) list for a current block. The video coding device derive an HMVP candidate from a previously coded block. The HMVP candidate may include motion information associated with a neighboring block of the current block, one or more reference indices, and a bi-prediction weight index. The video coding device may add the HMVP candidate to the HMVP list for motion compensated prediction of a motion vector associated with the current block. The video coding device use one HMVP selected from the HMVP list to perform motion compensated prediction of the current block. The motion compensated prediction may be performed using the motion information associated with the neighboring block of the current block, the one or more reference indices, and the bi-prediction weight index.

Claims (101)

1. A device for video decoding, the device comprising:

a processor configured to:

generate a candidate list for performing motion compensated prediction associated with a current block, wherein the current block is partitioned into a triangular first partition and a second partition;

add at least one of a spatial candidate or a temporal candidate to the candidate list;

derive a history-based motion vector prediction (HMVP) candidate from a previously coded block;

add the HMVP candidate to the candidate list; and

decode the current block that comprises the triangular first partition and the second partition based on the candidate list.

2. The device of claim 1 , wherein the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate.

3. The device of claim 1 , wherein the processor is further configured to:

interleave the HMVP candidate and at least one of the spatial candidate or the temporal candidate.

4. The device of claim 1 , wherein the processor is further configured to:

identify a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is associated with the triangular first partition and the second candidate is associated with the second partition, and

the current block is decoded based on the first candidate being associated with the triangular first partition and the second candidate being associated with the second partition.

5. The device of claim 1 , wherein the processor is further configured to:

identify a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is decoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

6. The device of claim 1 , wherein the processor is further configured to:

identify a merge index associated with the triangular first partition and the second partition; and

identify a first candidate from the candidate list and a second candidate from the candidate list based on the merge index, wherein:

the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate,

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is decoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

7. The device of claim 1 , wherein the HMVP candidate comprises motion information and a reference index.

8. The device of claim 1 , wherein the second partition is partitioned at an angle and has different dimensions than the triangular first partition.

9. The device of claim 1 , wherein the candidate list is a uni-prediction motion vector candidate list.

10. The device of claim 1 , wherein the candidate list is a uni-prediction motion vector candidate list, the spatial candidate is a first spatial candidate, the temporal candidate is a first temporal candidate, the HMVP candidate is a first HMVP candidate, and the processor is further configured to:

determine at least one of the first spatial candidate or the first temporal candidate and the first HMVP candidate are associated with a first set of motion vectors;

determine at least one of a second spatial candidate or a second temporal candidate and a second HMVP candidate are associated with a second set of motion vectors;

add at least one of the first spatial candidate or the first temporal candidate to the uni-prediction motion vector candidate list;

add the first HMVP candidate to the uni-prediction motion vector candidate list;

add at least one of the second spatial candidate or the second temporal candidate to the uni-prediction motion vector candidate list; and

add the second HMVP candidate to the uni-prediction motion vector candidate list.

11. A method for video decoding, the method comprising:

generating a candidate list for performing motion compensated prediction associated with a current block, wherein the current block is partitioned into a triangular first partition and a second partition;

adding at least one of a spatial candidate or a temporal candidate to the candidate list;

deriving a history-based motion vector prediction (HMVP) candidate from a previously coded block;

adding the HMVP candidate to the candidate list; and

decoding the current block that comprises the triangular first partition and the second partition based on the candidate list.

12. The method of claim 11 , wherein the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate.

13. The method of claim 11 , further comprising:

interleaving the HMVP candidate and at least one of the spatial candidate or the temporal candidate.

14. The method of claim 11 , further comprising:

identifying a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is associated with the triangular first partition and the second candidate is associated with the second partition, and

the current block is decoded based on the first candidate being associated with the triangular first partition and the second candidate being associated with the second partition.

15. The method of claim 11 , further comprising:

identifying a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is decoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

16. The method of claim 11 , further comprising:

identifying a merge index associated with the triangular first partition and the second partition; and

identifying a first candidate from the candidate list and a second candidate from the candidate list based on the merge index, wherein:

the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate,

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is decoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

17. The method of claim 11 , wherein the HMVP candidate comprises motion information and a reference index.

18. The method of claim 11 , wherein the second partition is partitioned at an angle and has different dimensions than the triangular first partition.

19. The method of claim 11 , wherein the candidate list is a uni-prediction motion vector candidate list.

20. A device for video encoding, the device comprising:

a processor configured to:

generate a candidate list for performing motion compensated prediction associated with a current block, wherein the current block is partitioned into a triangular first partition and a second partition;

add at least one of a spatial candidate or a temporal candidate to the candidate list;

derive a history-based motion vector prediction (HMVP) candidate from a previously coded block;

add the HMVP candidate to the candidate list; and

encode the current block that comprises the triangular first partition and the second partition based on the candidate list.

21. The device of claim 20 , wherein the processor is further configured to:

identify a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is associated with the triangular first partition and the second candidate is associated with the second partition, and

the current block is encoded based on the first candidate being associated with the triangular first partition and the second candidate being associated with the second partition.

22. The device of claim 20 , wherein the processor is further configured to:

identify a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is encoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

23. The device of claim 20 , wherein the processor is further configured to:

identify a merge index associated with the triangular first partition and the second partition; and

identify a first candidate from the candidate list and a second candidate from the candidate list based on the merge index, wherein:

the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate,

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is encoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

24. A method for video encoding, the method comprising:

generating a candidate list for performing motion compensated prediction associated with a current block, wherein the current block is partitioned into a triangular first partition and a second partition;

adding at least one of a spatial candidate or a temporal candidate to the candidate list;

deriving a history-based motion vector prediction (HMVP) candidate from a previously coded block;

adding the HMVP candidate to the candidate list; and

encoding the current block that comprises the triangular first partition and the second partition based on the candidate list.

25. The method of claim 24 , further comprising:

identifying a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is associated with the triangular first partition and the second candidate is associated with the second partition, and

the current block is encoded based on the first candidate being associated with the triangular first partition and the second candidate being associated with the second partition.

26. The method of claim 24 , further comprising:

identifying a first candidate from the candidate list and a second candidate from the candidate list, wherein:

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is encoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

27. The method of claim 24 , further comprising:

identifying a merge index associated with the triangular first partition and the second partition; and

identifying a first candidate from the candidate list and a second candidate from the candidate list based on the merge index, wherein:

the HMVP candidate is added to the candidate list after at least one of the spatial candidate or the temporal candidate,

the first candidate is the HMVP candidate associated with the triangular first partition and the second candidate is the spatial candidate or the temporal candidate associated with the second partition, and

the current block is encoded based on the HMVP candidate being associated with the triangular first partition and the spatial candidate or the temporal candidate being associated with the second partition.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 4, 2024
From: XIU, XIAOYU; HE, YUWEN; LUO, JIANCONG
To: VID SCALE, INC.
Reel/Frame 069480/0788 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: VID SCALE, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 068284/0031 →
Continuity (3)
Continuation 17419361
Provisional Application 62786429 · Dec 29, 2018
Related Publication 20240089427A1 · Mar 14, 2024
References Cited (43)
US 9503720B2 · Chen et al. · 2016 [cited by applicant]
US 20110194609A1 · Rusert et al. · 2011 [cited by applicant]
US 20110200107A1 · Ryu · 2011 [cited by examiner]
US 20140044180A1 · Chen · 2014 [cited by examiner]
US 20140071235A1 · Zhang et al. · 2014 [cited by applicant]
US 20140161186A1 · Zhang et al. · 2014 [cited by applicant]
US 20160219278A1 · Chen et al. · 2016 [cited by applicant]
US 20200186818A1 · Li · 2020 [cited by examiner]
US 20200204807A1 · Ye · 2020 [cited by examiner]
US 20210243470A1 · Solovyev · 2021 [cited by examiner]
US 20220060687A1 · Jang · 2022 [cited by examiner]
CN 104662909A · 2015 [cited by applicant]
CN 107211156A · 2017 [cited by applicant]
GB 2580326A · 2020 [cited by examiner]
JP 2021533696A · 2021 [cited by applicant]
JP 2022505771A · 2022 [cited by applicant]
JP 2022506717A · 2022 [cited by applicant]
RU 2574831C2 · 2016 [cited by applicant]
RU 2624560C2 · 2017 [cited by applicant]
WO 2013077659A1 · 2013 [cited by applicant]
WO 2020030187A1 · 2020 [cited by applicant]
WO 2020085954A1 · 2020 [cited by applicant]
WO 2020123218A1 · 2020 [cited by applicant]
WO WO2020126262A1 · 2020 [cited by examiner]
Chen, Jianle; Ye, Yan; Hwan Kim, Sueng; “Test Model 3 of Versatile Video Coding (VTM 3)”, JVET-L1002-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 12, 2018 (Year: 2018). [cited by examiner]
“VTM-3.0 Reference Software”, Available at <https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-3.0>, pp. 1-2. [cited by applicant]
Alshina et al., “Known Tools Performance Investigation for Next Generation Video Coding”, VCEG-AZ05, Samsung Electronics, ITU—Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Gr… [cited by applicant]
Bross et al., “High Efficiency Video Coding (HEVC) Text Specification Draft 10 (for FDIS & Consent)”, JCTVC-L1003_V1, Editor, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29… [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 3)”, JVET-L1001-V7, Editors, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 223 pages. [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 3)”, JVET-L1001-v9, Editors, JVET of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 233 pages. [cited by applicant]
Chen et al., “Coding Tools Investigation for Next Generation Video Coding”, Qualcomm Incorporated, COM 16-C 806-E, International Telecommunication Union, Telecommunication Standardization Sector, Jan. 2015, pp. 1-7. [cited by applicant]
Chen et al., “Generalized Bi-Prediction Method for Future Video Coding”, IEEE 2016 Picture Coding Symposium (PCS), Dec. 2016, 5 pages. [cited by applicant]
ITU-T, “Advanced Video Coding for Generic Audiovisual Services”, ITU-T Recommendation H.264, Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving Video, Nov. 2007, 563 pa… [cited by applicant]
Karczewicz et al., “Report of AHG1 on Coding Efficiency Improvements”, VCEG-AZ01, Qualcomm, Samsung, ITU—Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meet… [cited by applicant]
Ohm et al., “Report of AHG on Future Video Coding Standardization Challenges”, AHG, ISO/IEC JTC1/SC29/WG11 MPEG2014/M36782, Warsaw, Poland, Jun. 2015, 4 pages. [cited by applicant]
Segall et al., “Joint Call for Proposals on Video Compression with Capability Beyond HEVC”, JVET-H1002 (V6), Editors, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 8th Meeting: M… [cited by applicant]
SMPTE, “VC-1 Compressed Video Bitstream Format and Decoding Process”, SMPTE 421M, Apr. 2006, 493 pages. [cited by applicant]
Su et al., “CE4-related Generalized Bi-Prediction Improvements Combined from JVET-L0197 and JVET-L0296”, JVET-L0646-v5, MediaTek Inc., InterDigital Communications, Inc., Joint Video Experts Team (JVET) of ITU-T SG 16 WP… [cited by applicant]
Tourapis et al., “H.264/14496-10 AVC Reference Software Manual”, JVT-AE010, Dolby Laboratories Inc., Fraunhofer-Institute HHI, Microsoft Corporation, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC2… [cited by applicant]
Zhang et al., “CE4: History-based Motion Vector Prediction (Test 4.4.7)”, JVET-L0266-v1, Bytedance Inc., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, Oct. 3… [cited by applicant]
Zhang et al., “CE4-Related: History-Based Motion Vector Prediction”, JVET-K0104-V5, Bytedance Inc., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Jul. 10… [cited by applicant]
Zhang et al., “Non-CE4: Harmonization between HMVP and GBi”, JVET-M0264, Peking University, Bytedance Inc. and InterDigital Communications, Inc., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC … [cited by applicant]
Park et al., “CE4-related: History-Based Motion Vector Prediction Considering Parallel Processing”, JVET-L0158, LG Electronics Inc., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12t… [cited by applicant]