IP Library Granted Patent US 12,425,576
Granted Patent B2
US 12,425,576 · App. 18/663,914 · Granted Sep 23, 2025

Methods for constructing a merge candidate list

Inventors: Zhao Wang (Beijing, CN); Yan Ye (San Mateo, CA); Jiancong Luo (San Mateo, CA)
Assignee: Alibaba Group Holding Limited
H04N19/105H04N19/159H04N19/176H04N19/423
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,425,576
App. No.
18/663,914
Granted
Sep 23, 2025
Kind
B2
Abstract

A video processing method is provided. The method includes inserting a set of spatial merge candidates to a first merge candidate list of a coding block, wherein the first merge candidate list is constructed by: adding a top neighboring block B 1 to the first merge candidate list when the top neighboring block B 1 is available; adding a left neighboring block A 1 to the first merge candidate list when the left neighboring block A 1 is available; adding a top neighboring block B 0 to the first merge candidate list when the top neighboring block B 0 is available; adding a left neighboring block A 0 to the first merge candidate list when the left neighboring block A 0 is available; and adding an above-left neighboring block B 2 to the first merge candidate list when the above-left neighboring block B 2 is available.

Claims (57)

1. A method of encoding a video sequence into a bitstream, the method comprising:

receiving a video sequence;

encoding one or more pictures of the video sequence; and

generating a bitstream, wherein the encoding comprises:

inserting a set of spatial merge candidates to a merge candidate list of a coding block, wherein:

when a first coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a first construction order;

when a second coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a second construction order, the first construction order being different from the second construction order; and

adding to the merge candidate list a history-based motion vector predictor (HMVP) from a first-in-first-out (FIFO) table, wherein

motion information of a previously coded block is stored in the FIFO table and used as the motion vector predictor for a current coding unit; or

motion information associated with a non-subblock inter-coded coding unit is added to the last entry of the FIFO table as a new HMVP candidate.

2. The method according to claim 1 , wherein the first coding mode and second coding mode are two different modes selected from:

a regular merge mode, a merge mode with motion vector difference (MMVD), and a triangle partition mode (TPM).

3. The method according to claim 2 , wherein when the MMVD is selected, the method further comprising:

selecting a merge candidate from the merge candidate list;

refining the merge candidate by signaled motion vector difference (MVD) information; and

signaling a merge candidate flag to specify which MMVD candidate in the merge candidate list is used as a base motion vector.

4. The method according to claim 2 , wherein when the TPM is selected, a coding unit is split evenly into two triangle-shaped partitions, using at least one of a diagonal split or an anti-diagonal split.

5. The method according to claim 1 , further comprising:

adding to the merge candidate list at least one of: a temporal merge candidate from collocated coding units, a pairwise average candidate, or a zero motion vector.

6. The method according to claim 5 , wherein when the pairwise average candidate is added, the pairwise average candidate is generated by averaging pairs of candidates in the merge candidate list and is added to the merge candidate list after one or more HMVPs being added to the merge candidate list in response to the merge candidate list not being full.

7. The method according to claim 5 , wherein when the zero motion vector is added, the zero motion vector is inserted in an end of the merge candidate list until a maximum merge candidate number is reached.

8. A method of decoding a bitstream to output one or more pictures for a video stream, the method comprising:

receiving a bitstream; and

decoding, using coded information of the bitstream, one or more pictures, wherein the decoding comprises:

inserting a set of spatial merge candidates to a merge candidate list of a coding block, wherein:

when a first coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a first construction order; and

when a second coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a second construction order, the first construction order being different from the second construction order; and

adding to the merge candidate list a history-based motion vector predictor (HMVP) from a first-in-first-out (FIFO) table, wherein

motion information of a previously coded block is stored in the FIFO table and used as the motion vector predictor for a current coding unit; or

motion information associated with a non-subblock inter-coded coding unit is added to the last entry of the FIFO table as a new HMVP candidate.

9. The method according to claim 8 , wherein the first coding mode and second coding mode are two different modes selected from:

a regular merge mode, a merge mode with motion vector difference (MMVD), and a triangle partition mode (TPM).

10. The method according to claim 9 , wherein when the MMVD is selected, the method further comprising:

selecting a merge candidate from the merge candidate list;

refining the merge candidate by decoded motion vector difference (MVD) information; and

decoding a merge candidate flag to specify which MMVD candidate in the merge candidate list is used as a base motion vector.

11. The method according to claim 9 , wherein when the TPM is selected, a coding unit is split evenly into two triangle-shaped partitions, using at least one of a diagonal split or an anti-diagonal split.

12. The method according to claim 8 , further comprising:

adding to the merge candidate list at least one of: a temporal merge candidate from collocated coding units, a pairwise average candidate, or a zero motion vector.

13. The method according to claim 12 , wherein when the pairwise average candidate is added, the pairwise average candidate is generated by averaging pairs of candidates in the merge candidate list and is added to the merge candidate list after one or more HMVPs being added to the merge candidate list in response to the merge candidate list not being full.

14. The method according to claim 12 , wherein when the zero motion vector is added, the zero motion vector is inserted in an end of the merge candidate list until a maximum merge candidate number is reached.

15. A non-transitory computer readable storage medium storing a bitstream of a video for processing according to a method comprising:

inserting a set of spatial merge candidates to a merge candidate list of a coding block, wherein:

when a first coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a first construction order;

when a second coding mode is applied to the coding block, the set of spatial merge candidates are inserted according to a second construction order, the first construction order being different from the second construction order; and

adding to the merge candidate list a history-based motion vector predictor (HMVP) from a first-in-first-out (FIFO) table, wherein

motion information of a previously coded block is stored in the FIFO table and used as the motion vector predictor for a current coding unit; or

motion information associated with a non-subblock inter-coded coding unit is added to the last entry of the FIFO table as a new HMVP candidate.

16. The non-transitory computer readable medium according to claim 15 , wherein the first coding mode and second coding mode are two different modes selected from: a regular merge mode, a merge mode with motion vector difference (MMVD), and a triangle partition mode (TPM).

17. The non-transitory computer readable medium according to claim 16 , wherein when the MMVD is selected, the method further comprises:

selecting a merge candidate from the merge candidate list;

refining the merge candidate by signaled motion vector difference (MVD) information; and

signaling a merge candidate flag to specify which MMVD candidate in the merge candidate list is used as a base motion vector.

18. The non-transitory computer readable medium according to claim 16 , wherein when the TPM is selected, a coding unit is split evenly into two triangle-shaped partitions, using at least one of a diagonal split or an anti-diagonal split.

19. The non-transitory computer readable medium according to claim 15 , wherein the method further comprises:

adding to the merge candidate list at least one of: a temporal merge candidate from collocated coding units, a pairwise average candidate, and a zero motion vector.

20. The non-transitory computer readable medium according to claim 19 , wherein when the pairwise average candidate is added, the pairwise average candidate is generated by averaging pairs of candidates in the merge candidate list and is added to the merge candidate list after one or more HMVPs being added to the merge candidate list in response to the merge candidate list not being full.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2026
From: ALIBABA INNOVATION PRIVATE LIMITED
To: HFI INNOVATION INC.
Reel/Frame 075499/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2026
From: ALIBABA GROUP HOLDING LIMITED
To: ALIBABA INNOVATION PRIVATE LIMITED
Reel/Frame 074832/0914 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2024
From: WANG, ZHAO; YE, YAN; LUO, JIANCONG
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 067512/0455 →
Continuity (4)
Continuation 18052011 · Nov 2, 2022
Continuation 16996242 · Aug 18, 2020
Provisional Application 62902790 · Sep 19, 2019
Related Publication 20240305773A1 · Sep 12, 2024
References Cited (55)
US 9338459B2 · Oh et al. · 2016 [cited by applicant]
US 10194133B2 · Bang · 2019 [cited by examiner]
US 10491902B1 · Xu · 2019 [cited by examiner]
US 10812810B2 · Ye · 2020 [cited by examiner]
US 20130287116A1 · Helle · 2013 [cited by examiner]
US 20150117543A1 · Hong · 2015 [cited by examiner]
US 20150131733A1 · Kim et al. · 2015 [cited by applicant]
US 20170310988A1 · Lin · 2017 [cited by examiner]
US 20170347093A1 · Yu · 2017 [cited by examiner]
US 20180242024A1 · Chen · 2018 [cited by examiner]
US 20180249154A1 · Chuang · 2018 [cited by examiner]
US 20180359483A1 · Chen · 2018 [cited by examiner]
US 20190182489A1 · Xu et al. · 2019 [cited by applicant]
US 20190246118A1 · Ye · 2019 [cited by examiner]
US 20200186825A1 · Rusanovskyy · 2020 [cited by examiner]
US 20200275093A1 · Lin · 2020 [cited by examiner]
US 20200280728A1 · Ye · 2020 [cited by examiner]
US 20200413081A1 · Rusanovskyy · 2020 [cited by examiner]
US 20210014484A1 · Kotra · 2021 [cited by examiner]
US 20210037238A1 · Park · 2021 [cited by examiner]
US 20210105499A1 · Lee · 2021 [cited by examiner]
US 20210306661A1 · Ahn · 2021 [cited by examiner]
US 20210392337A1 · Lim · 2021 [cited by examiner]
US 20220070444A1 · Takehara et al. · 2022 [cited by applicant]
US 20220070448A1 · Kim · 2022 [cited by examiner]
US 20220078407A1 · Zhao · 2022 [cited by examiner]
US 20220078470A1 · Lee · 2022 [cited by examiner]
US 20220109835A1 · Lee · 2022 [cited by examiner]
US 20220116594A1 · Park · 2022 [cited by examiner]
US 20220182614A1 · Jang · 2022 [cited by examiner]
CN 103931196A · 2014 [cited by applicant]
JP 2013016931A · 2013 [cited by applicant]
JP 2016054551A · 2016 [cited by applicant]
JP 2022033109A · 2022 [cited by applicant]
WO 2020184459A1 · 2020 [cited by applicant]
“Requirements for a Future Video Coding Standard v5,” International Organisation for Standardisation, Torino, IT, 14 pages (2017). [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 5),” JVET-N1001-v8, 14 [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 6), ”JVET-O2001-vE, 15 [cited by applicant]
Chen et al., Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), JVET-N1002-v2, 14 [cited by applicant]
Chen et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” JVET-G1001-v1, 7 [cited by applicant]
International Telecommunications Union “Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video”, ITU-T Telecommunication Standardization Sector of ITU, Apr. 2013, 317 … [cited by applicant]
JEM, https://jvet.hhi.fraunhofer_de/svn/svn_HMJEMSoftware. [cited by applicant]
PCT International Search Report and Written Opinion mailed Nov. 18, 2020, issued in corresponding International Application No. PCT/US2020/046799 (7 pgs.). [cited by applicant]
Sullivan et al., “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEE Transactions on Circuits and Systems for Video Technology, vol. 22, No. 12, pp. 1649-1668 (2012). [cited by applicant]
European Patent Office Communication issued for Application No. 20866064.7 the Supplementary European Search Report (Art. 153(7) EPC) and the European search opinion dated Oct. 20, 2022, 11 pages. [cited by applicant]
Chen et al., “Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6),” JVETO2002-v2, 15 [cited by applicant]
Li et al., “CE4-ranking based spatial merge candidate list for inter prediction (Test CE4.4.14),” JVET-L0318, 12 [cited by applicant]
Zhang et al., “History-based Motion Vector Prediction in Versatile Video Coding,” 2019 Data Compression Conference (DCC), pp. 43-52. [cited by applicant]
First Office Action issued in corresponding Chinese Application No. 202080065957.8 on Nov. 30, 2024 (7 pages). [cited by applicant]
Japanese Office Action issued in corresponding Japanese Application No. 2022-514661 on Sep. 2, 2024 (15 pages). [cited by applicant]
Japanese Office Action issued in corresponding Japanese Application No. 2022-514661 on Feb. 26, 2025 (10 pages). [cited by applicant]
Japanese Search Report issued in corresponding Japanese Application No. 2022-514661 on Apr. 23, 2024 (30 pages). [cited by applicant]
Kim et al., “Non-CE9: reordering of merge candidate,” JCTVC-G396, 7th Meeting: Geneva, CH, Nov. 19-30, 2011, 16 pages. [cited by applicant]
Wang et al., “Non-CE4: Construction of spatial merge candidates,” JVET-P0325-v3, 16th Meeting: Geneva, CH, 1-11 Oct. 19, 4 pages. [cited by applicant]
Xu et al., “CE4: Candidate List Reordering (Test 4.4.13),” JVET-L0186-v2, 12th Meeting: Macao, CN, Oct. 3-12, 2018, 7 pages. [cited by applicant]