IP Library › Granted Patent US 12,395,622
Granted Patent B2
US 12,395,622 · App. 17/479,482 · Granted Aug 19, 2025

Position-dependent intra-inter prediction combination in video coding

Inventors: Geert Van der Auwera (Del Mar, CA); Luong Pham Van (San Diego, CA); Adarsh Krishnan Ramasubramonian (Irvine, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM INCORPORATED
H04N19/105H04N19/132H04N19/136H04N19/159H04N19/176H04N19/423
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,395,622
App. No.
17/479,482
Granted
Aug 19, 2025
Kind
B2
Abstract

An example device for coding (encoding or decoding) video data includes a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a first number of neighboring blocks to a current block of the video data that are intra-predicted; determine a second number of the neighboring blocks that are inter-predicted; determine a first weight value to be applied to intra-prediction samples of an intra-prediction block for the current block; determine a second weight value to be applied to inter-prediction samples of an inter-prediction block for the current block; generate a prediction block for the current block as a weighted combination of the intra-prediction block to which the first weight value is applied and the inter-prediction block to which the second weight value is applied; and code the current block using the prediction block.

Claims (99)

1. A method of decoding video data, the method comprising:

forming an inter-prediction block for a current luminance block of video data;

forming an intra-prediction block for the current luminance block of the video data;

determining whether a first neighboring block to the current luminance block is intra-prediction coded, wherein the first neighboring block comprises an above-neighboring block;

determining whether a second neighboring block to the current luminance block is intra-prediction coded, wherein the second neighboring block comprises a left-neighboring block;

determining, based on a combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a first weight;

determining, based on the combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a second weight;

applying the first weight to the inter-prediction block to form a first weighted inter-prediction block and applying the second weight to the intra-prediction block to form a second weighted intra-prediction block;

combining the first weighted inter-prediction block and the second weighted intra-prediction block to form a prediction block for the current luminance block; and

decoding the current luminance block using the prediction block.

2. The method of claim 1 , wherein:

based on the first neighboring block and the second neighboring block each being intra-predicted, determining the first weight and the second weight comprises determining the first weight to be equal to 3 and determining the second weight to be equal to 1;

based on one of the first neighboring block or the second neighboring block being intra-predicted and the other of the first neighboring block or the second neighboring block being inter-predicted, determining the first weight and the second weight comprises determining the first weight to be equal to 2 and determining the second weight to be equal to 2 ; or

based on the first neighboring block and the second neighboring block each being inter-predicted, determining the first weight and the second weight comprises determining the first weight to be equal to 1 and determining the second weight to be equal to 3.

3. The method of claim 1 , wherein:

based on the first neighboring block and the second neighboring block each being determined to be inter-prediction coded, determining the first weight and the second weight comprises determining the first weight to be greater than the second weight.

4. The method of claim 1 , wherein determining the first weight and the second weight comprises determining the first weight and the second weight based on dimensions of the current luminance block.

5. The method of claim 1 , wherein determining the first weight and the second weight comprises determining the first weight and the second weight based on inter mode information or motion information.

6. The method of claim 1 , wherein decoding the current luminance block comprises:

decoding a residual block for the current luminance block; and

combining samples of the residual block with samples of the prediction block.

7. The method of claim 1 , further comprising encoding the current luminance block prior to decoding the current luminance block, wherein encoding the current luminance block comprises:

subtracting samples of the prediction block from samples of the current luminance block to form a residual block; and

encoding the residual block.

8. A device for decoding video data, the device comprising:

a memory configured to store video data; and

one or more processors implemented in circuitry and configured to:

form an inter-prediction block for a current luminance block of video data;

form an intra-prediction block for the current luminance block of the video data;

determine whether a first neighboring block to the current luminance block is intra-prediction coded, wherein the first neighboring block comprises an above-neighboring block;

determine whether a second neighboring block to the current luminance block is intra-prediction coded, wherein the second neighboring block comprises a left-neighboring block;

determine, based on a combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a first weight;

determine, based on the combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a second weight:

apply the first weight to the inter-prediction block to form a first weighted inter-prediction block and apply the second weight to the intra-prediction block to form a second weighted intra-prediction block;

combine the first weighted inter-prediction block and the second weighted intra-prediction block to form a prediction block for the current luminance block; and

decode the current luminance block using the prediction block.

9. The device of claim 8 , wherein the one or more processors are configured to:

based on the first neighboring block and the second neighboring block each being intra-predicted, determine the first weight to be equal to 3 and determine the second weight to be equal to 1;

based on one of the first neighboring block or the second neighboring block being intra-predicted and the other of the first neighboring block or the second neighboring block being inter-predicted, determine the first weight to be equal to 2 and determine the second weight to be equal to 2; or

based on the first neighboring block and the second neighboring block each being inter-predicted, determine the first weight to be equal to 1 and determine the second weight to be equal to 3.

10. The device of claim 8 , wherein the one or more processors are configured to:

based on the first neighboring block and the second neighboring block each being determined to be inter-prediction coded, determine the first weight to be greater than the second weight; and

based on the first neighboring block and the second neighboring block each being determined to be intra-prediction coded, determine the first weight to be less than the second weight.

11. The device of claim 8 , wherein the one or more processors are further configured to determine the first weight and the second weight based on dimensions of the current luminance block.

12. The device of claim 8 , wherein the one or more processors are further configured to determine the first weight and the second weight based on inter mode information or motion information.

13. The device of claim 8 , wherein to decode the current luminance block, the one or more processors are configured to:

decode a residual block for the current luminance block; and

combine samples of the residual block with samples of the prediction block.

14. The device of claim 8 , wherein the one or more processors are further configured to encode the current luminance block prior to decoding the current luminance block, wherein to encode the current luminance block, the one or more processors are configured to:

subtract samples of the prediction block from samples of the current luminance block to form a residual block; and

encode the residual block.

15. The device of claim 8 , further comprising a display configured to display the decoded video data.

16. The device of claim 8 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

17. A non-transitory, computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

form an inter-prediction block for a current luminance block of video data;

form an intra-prediction block for the current luminance block of the video data;

determine whether a first neighboring block to the current luminance block is intra-prediction coded, wherein the first neighboring block comprises an above-neighboring block;

determine whether a second neighboring block to the current luminance block is intra-prediction coded, wherein the second neighboring block comprises a left-neighboring block;

determine, based on a combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a first weight;

determine, based on the combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a second weight;

apply the first weight to the inter-prediction block to form a first weighted inter-prediction block and apply the second weight to the intra-prediction block to form a second weighted intra-prediction block;

combine the first weighted inter-prediction block and the second weighted intra-prediction block to form a prediction block for the current luminance block; and

decode the current luminance block using the prediction block.

18. The non-transitory, computer-readable storage medium of claim 17 , wherein the instructions that cause the processor to determine the first weight and the second weight comprise instructions that cause the processor to:

based on the first neighboring block and the second neighboring block each being intra-predicted, determine the first weight to be equal to 3 and determine the second weight to be equal to 1;

based on one of the first neighboring block or the second neighboring block being intra-predicted and the other of the first neighboring block or the second neighboring block being inter-predicted, determine the first weight to be equal to 2 and determine the second weight to be equal to 2; or

based on the first neighboring block and the second neighboring block each being inter-predicted, determine the first weight to be equal to 1 and determining the second weight to be equal to 3.

19. The non-transitory, computer-readable storage medium of claim 17 , wherein the instructions that cause the processor to determine the first weight and the second weight comprise instructions that cause the processor to:

based on the first neighboring block and the second neighboring block each being determined to be inter-prediction coded, determine the first weight to be greater than the second weight; and

based on the first neighboring block and the second neighboring block each being determined to be intra-prediction coded, determine the first weight to be less than the second weight.

20. The non-transitory, computer-readable storage medium of claim 17 , wherein the instructions that cause the processor to decode the current luminance block comprise instructions that cause the processor to:

decode a residual block for the current luminance block; and

combine samples of the residual block with samples of the prediction block.

21. The non-transitory, computer-readable storage medium of claim 17 , further comprising instructions that cause the processor to encode the current luminance block prior to decoding the current luminance block, wherein the instructions that cause the processor to encode the current luminance block comprise instructions that cause the processor to:

subtract samples of the prediction block from samples of the current luminance block to form a residual block; and

encode the residual block.

22. A device for decoding video data, the device comprising:

means for forming an inter-prediction block for a current luminance block of video data;

means for forming an intra-prediction block for the current luminance block of the video data;

means for determining whether a first neighboring block to the current luminance block is intra-prediction coded, wherein the first neighboring block comprises an above-neighboring block;

means for determining whether a second neighboring block to the current luminance block is intra-prediction coded, wherein the second neighboring block comprises a left-neighboring block;

means for determining, based on a combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a first weight;

means for determining, based on the combination of the determination of whether the first neighboring block is intra prediction coded and the determination of whether the second neighboring block is intra-prediction coded, a second weight;

means for applying the first weight to the inter-prediction block to form a first weighted inter-prediction block and the second weight to the intra-prediction block to form a second weighted intra-prediction block;

means for combining the first weighted inter-prediction block and the second weighted intra-prediction block to form a prediction block for the current luminance block; and

means for decoding the current luminance block using the prediction block.

23. The device of claim 22 , wherein the means for determining the first weight and the second weight comprises:

means for determining the first weight to be equal to 3 and determining the second weight to be equal to 1 based on the first neighboring block and the second neighboring block each being intra-predicted;

means for determining the first weight to be equal to 2 and determining the second weight to be equal to 2 based on one of the first neighboring block or the second neighboring block being intra-predicted and the other of the first neighboring block or the second neighboring block being inter-predicted; and

means for determining the first weight to be equal to 1 and determining the second weight to be equal to 3 based on the first neighboring block and the second neighboring block each being inter-predicted.

24. The device of claim 22 , wherein the means for determining the first weight and the second weight comprises:

means for determining the first weight to be greater than the second weight based on the first neighboring block and the second neighboring block each being determined to be inter-prediction coded; and

means for determining the first weight to be less than the second weight based on the first neighboring block and the second neighboring block each being determined to be intra-prediction coded.

25. The device of claim 22 , wherein the means for decoding the current luminance block comprises:

means for decoding a residual block for the current luminance block; and

means for combining samples of the residual block with samples of the prediction block.

26. The device of claim 22 , further comprising means for encoding the current luminance block prior to decoding the current luminance block, wherein the means for encoding the current luminance block comprises:

means for subtracting samples of the prediction block from samples of the current luminance block to form a residual block; and

means for encoding the residual block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2021
From: VAN DER AUWERA, GEERT; PHAM VAN, LUONG; RAMASUBRAMONIAN, ADARSH KRISHNAN; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 057533/0316 →
Continuity (4)
Continuation 16684440 · Nov 14, 2019
Provisional Application 62784140 · Dec 21, 2018
Provisional Application 62768655 · Nov 16, 2018
Related Publication 20220007010A1 · Jan 6, 2022
References Cited (68)
US 9374578B1 · Mukherjee et al. · 2016 [cited by applicant]
US 9654785B2 · Chien et al. · 2017 [cited by applicant]
US 11006109B2 · Jang et al. · 2021 [cited by applicant]
US 11070815B2 · Lai et al. · 2021 [cited by applicant]
US 11463689B2 · Zhao et al. · 2022 [cited by applicant]
US 20050013360A1 · Kim et al. · 2005 [cited by applicant]
US 20070047648A1 · Tourapis et al. · 2007 [cited by applicant]
US 20080240245A1 · Lee et al. · 2008 [cited by applicant]
US 20120307899A1 · Kim · 2012 [cited by examiner]
US 20140010283A1 · Lim · 2014 [cited by examiner]
US 20140072041A1 · Seregin et al. · 2014 [cited by applicant]
US 20160373770A1 · Zhao et al. · 2016 [cited by applicant]
US 20170251213A1 · Ye et al. · 2017 [cited by applicant]
US 20170347102A1 · Panusopone et al. · 2017 [cited by applicant]
US 20180249156A1 · Heo et al. · 2018 [cited by applicant]
US 20180278942A1 · Zhang · 2018 [cited by examiner]
US 20180288410A1 · Park · 2018 [cited by examiner]
US 20180324417A1 · Karczewicz et al. · 2018 [cited by applicant]
US 20180359470A1 · Lee et al. · 2018 [cited by applicant]
US 20180376148A1 · Zhang et al. · 2018 [cited by applicant]
US 20190215521A1 · Chuang et al. · 2019 [cited by applicant]
US 20190306513A1 · Van Der Auwera et al. · 2019 [cited by applicant]
US 20200007895A1 · Van Der Auwera et al. · 2020 [cited by applicant]
US 20200077089A1 · Lee · 2020 [cited by examiner]
US 20200154114A1 · Zhao et al. · 2020 [cited by applicant]
US 20200162728A1 · Van Der Auwera et al. · 2020 [cited by applicant]
US 20200162737A1 · Van Der Auwera et al. · 2020 [cited by applicant]
US 20200260091A1 · Pham Van et al. · 2020 [cited by applicant]
US 20220030226A1 · Lee et al. · 2022 [cited by applicant]
US 20220286680A1 · Van Der Auwera et al. · 2022 [cited by applicant]
CN 101877785A · 2010 [cited by applicant]
CN 102656887A · 2012 [cited by applicant]
CN 103402092A · 2013 [cited by applicant]
CN 107113425A · 2017 [cited by applicant]
CN 107979756A · 2018 [cited by applicant]
CN 108702515A · 2018 [cited by applicant]
WO 2012170812 · 2012 [cited by applicant]
WO 2016205712 · 2016 [cited by applicant]
WO 2016205718 · 2016 [cited by applicant]
WO 2018204664A1 · 2018 [cited by applicant]
Certified Copy of KR1020180112011, filed on Sep. 19, 2018, 161 Pages. [cited by applicant]
Certified Copy of KR1020180114690, filed on Sep. 27, 2018, 160 Pages. [cited by applicant]
Certified Copy of KR1020180137879, filed on Nov. 12, 2018, 171 Pages. [cited by applicant]
Bossen F., et al., “JEM Software Manual”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, Document: JCTVC-Software Manual, Retrieved on Aug. 3, 2016, pp. 1-29. [cited by applicant]
Bross B., et al., “Versatile Video Coding (Draft 3)”, 12th JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, CN, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16 WP 3), No. JVET-L1001-v9, Jan. … [cited by applicant]
Cha R., et al., “Improved Combined Inter-intra Prediction Using Spatial-Variant Weighted Coefficient,” International Conference on Multimedia and Expo (ICME), Jul. 11, 2011, pp. 1-5, XP031964687, DOI: 10.1109/ICME.2011.… [cited by applicant]
Chen J., et al., “Algorithm description for Versatile Video Coding and Test Model 2 (VTM 2)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-K1002-v2, 11th Meeting: Ljubljana, SI… [cited by applicant]
Chen J., et al., “Algorithm Description for Versatile Video Coding and Test Model 3 (VTM 3),” 12th JVET Meeting, Oct. 3, 2018- Oct. 12, 2018, MACAO, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T … [cited by applicant]
Chen J., et al., “Algorithm Description of Joint Exploration Test Model 1”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 1nd Meeting: Geneva, CH, Oct. 19-21, 2015, 1. JVET Meeti… [cited by applicant]
Chen J., et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM7)”, JVET-G1001-V1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11, No. N17055, 7th Meeting, Jul. 13, 2… [cited by applicant]
Chen Y., et al., “Description of SDR, HDR and 360 Degree Video Coding Technology Proposal by Qualcomm and Technicolor-Low and High Complexity Versions”, 10th Meeting, San Diego, US, Apr. 10-20, 2018 (Apr. 10, 2018-Apr. … [cited by applicant]
Chen Y., et al., “Joint Inter-Intra Prediction based on Mode-Variant and Edge-directed Weighting Approaches in Video Coding,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 4, 201… [cited by applicant]
Chiang M., et al., “CE1 0.1.1: Multi-hypothesis Prediction for Improving AMVP Mode, Skip or Merge Mode, and Intra Mode,” 12th JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, (The Joint Video Exploration Team of ISO/IEC… [cited by applicant]
De Rivaz P., et al., “AV1 Bitstream & Decoding Process Specification”, The Alliance for Open Media, Jun. 25, 2018, 677 Pages. [cited by applicant]
Ergen S., “ZigBee/IEEE 802.15.4 Summary”, Sep. 10, 2004, 37 Pages. [cited by applicant]
Hernández S-D., et al., “CE3: Line-based intra coding mode (Tests 2.1.1 and 2.1.2)”, 12th JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, CN, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16)… [cited by applicant]
IEEE Computer Society; “Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY)Specifications: Amendment 4: Enhancements for Very High Throughput for Operation in Bands below 6 Ghz”, IEEE Standard for… [cited by applicant]
International Search Report and Written Opinion—PCT/US2019/061855—ISA/EPO—Feb. 21, 2020-02. [cited by applicant]
ITU-T H.265, “Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving Video, High efficiency Video Coding”, The International Telecommunication Union, Dec. 2016, 664 Pages. [cited by applicant]
Liao (Panasonic) R-L., et al., “CE10.3.1.b: Triangular Prediction Unit Mode”, 12. JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, (The Joint Video Exploration Team of ISO/IEC JTC 1/SC 29/WG 11 and ITU-T SG.16), No. JVE… [cited by applicant]
Qualcomm: “Position Dependent Intra Prediction Combination”, ITU-T SG16 Meeting, (Dec. 10, 2015)—(Oct. 23, 2015), Geneva, No. T13-SG16-C-1046, Sep. 30, 2015 (Sep. 30, 2015), XP030100754, 5 pages, the whole document. [cited by applicant]
Van Der Auwera G., et al., “Extension of Simplified PDPC to Diagonal Intra Modes”, 10th JVET Meeting, Apr. 10, 2018-Apr. 20, 2018, San Diego, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16),… [cited by applicant]
Van L P., et al., “CE10: CIIP with Position-independent Weights (Test CE10-1.1)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, Document: … [cited by applicant]
Van L.P., (Qualcomm) et al., “CE10-related: Inter-Intra Prediction Combination”, 13. JVET Meeting; Jan. 9, 2019-Jan. 18, 2019; Marrakech; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16), No. … [cited by applicant]
Zhao L., et al., “CE3-related: Unification of Angular Intra Prediction for Square and Non-square Blocks,” 12, JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 … [cited by applicant]
Zhao L., et al., “Non-CE: Weighted Intra and Inter Prediction Mode”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O0537, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, Doc: JVE… [cited by applicant]
Zhao X., et al., “EE1 Related: Simplification and Extension of PDPC”, 8th JVET Meeting; Oct. 18, 2017-Oct. 25, 2017; Macau; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16); URL: http://pheni… [cited by applicant]
Zhao X., et al., “EE1 Related: Simplification and Extension of PDPC”, 8. JVET Meeting, JVET-H0057_r1, Oct. 18, 2017-Oct. 24, 2017, Macau, CN, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16),… [cited by applicant]