IP Library › Granted Patent US 12,341,990
Granted Patent B2
US 12,341,990 · App. 18/391,517 · Granted Jun 24, 2025

Method and apparatus for parametric, model-based, geometric frame partitioning for video coding

Inventors: Oscar Divorra Escoda (Barcelona, ES); Peng Yin (Ithaca, NY)
Assignee: InterDigital VC Holdings, Inc.
H04N19/57H04N19/117H04N19/119H04N19/126H04N19/13H04N19/146H04N19/156H04N19/159H04N19/176H04N19/189H04N19/44H04N19/50H04N19/507H04N19/543H04N19/61H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,341,990
App. No.
18/391,517
Granted
Jun 24, 2025
Kind
B2
Abstract

There are provided methods and apparatus for adaptive geometric partitioning for video encoding and decoding. An apparatus includes an encoder for encoding image data corresponding to pictures by adaptively partitioning at least portions of the pictures responsive to at least one parametric model. The at least one parametric model involves at least one of implicit and explicit formulation of at least one curve.

Claims (68)

1. An apparatus for video decoding, comprising:

at least one processor being configured to:

decode a parametric model for a current block of a picture, wherein said block is partitioned to include a first partition and a second partition according to said parametric model, wherein said at least one processor is configured to partition by:

associating values with respective pixels of said current block according to said parametric model, and

classifying said pixels of said current block into partitions according to said associated values, wherein a first value according to said parametric model indicates that a pixel of said current block is in said first partition, a second value indicates that a pixel of said current block is in said second partition, and a value in between said first and second values indicates that a pixel of said current block is in a partial surface;

predict motion vectors for said first and second partitions based on at least one of spatial neighboring blocks and temporal neighboring blocks;

decode a first motion vector for said first partition and a second motion vector for said second partition based on said predicted motion vectors;

predict said current block to form a predicted block by:

predicting pixels in said first partition using a first predictor,

predicting pixels in said second partition using a second predictor, and

predicting pixels in said partial surface based on a weighted linear average of said first predictor and said second predictor, wherein said first predictor is weighted by w(x,y) and said second predictor is weighted by (1−w(x,y)) to form said weighted linear average, wherein w(x, y) is said value from said parametric model for pixel at location (x,y); and

decode said current block based on said predicted block.

2. The apparatus of claim 1 , wherein each of said first and second motion vectors is predicted from an adaptively selected set of motion vectors from spatial and temporal neighboring blocks.

3. The apparatus of claim 1 , wherein said parametric model corresponds to a first-degree polynomial function.

4. The apparatus of claim 1 , wherein said at least one processor is further configured to:

predict said parametric model based on another parametric model of one or more spatial neighboring blocks, wherein said parametric model is decoded based on said predicted parametric model.

5. The apparatus of claim 4 , wherein said another parametric model is projected from a spatial neighboring block into said current block to predict said parametric model.

6. The apparatus of claim 1 , wherein said parametric model includes an angle parameter and a distance parameter.

7. The apparatus of claim 6 , wherein a line associated with said another parametric model is continued from a spatial neighboring block into said current block for predict said parametric model.

8. An apparatus for video encoding, comprising:

at least one processor being configured to:

encode a parametric model for a current block of a picture, wherein said block is partitioned to include a first partition and a second partition according to said parametric model, wherein said at least one processor is configured to partition by:

associating values with respective pixels of said current block according to said parametric model, and

classifying said pixels of said current block into partitions according to said associated values, wherein a first value according to said parametric model indicates that a pixel of said current block is in said first partition, a second value indicates that a pixel of said current block is in said second partition, and a value in between said first and second values indicates that a pixel of said current block is in a partial surface;

predict motion vectors for said first and second partitions based on at least one of spatial neighboring blocks and temporal neighboring blocks;

encode a first motion vector for said first partition and a second motion vector for said second partition based on said predicted motion vectors;

predict said current block to form a predicted block by:

predicting pixels in said first partition using a first predictor,

predicting pixels in said second partition using a second predictor, and

predicting pixels in said partial surface based on a weighted linear average of said first predictor and said second predictor, wherein said first predictor is weighted by w(x,y) and said second predictor is weighted by (1−w(x,y)) to form said weighted linear average, wherein w(x, y) is said value from said parametric model for pixel at location (x,y); and

encode said current block based on said predicted block.

9. The apparatus of claim 8 , wherein each of said first and second motion vectors is predicted from an adaptively selected set of motion vectors from spatial and temporal neighboring blocks.

10. The apparatus of claim 8 , wherein said parametric model corresponds to a first-degree polynomial function.

11. The apparatus of claim 8 , wherein said at least one processor is further configured to:

predict said parametric model based on another parametric model of one or more spatial neighboring blocks, wherein said parametric model is encoded based on said predicted parametric model.

12. The apparatus of claim 8 , wherein said another parametric model is projected from a spatial neighboring block into said current block to predict said parametric model.

13. The apparatus of claim 8 , wherein said parametric model includes an angle parameter and a distance parameter.

14. The apparatus of claim 13 , wherein a line associated with said another parametric model is continued from a spatial neighboring block into said current block for predict said parametric model.

15. A method of video decoding, comprising:

decoding a parametric model for a current block of a picture, wherein said block is partitioned to include a first partition and a second partition according to said parametric model, wherein said partitioning includes:

associating values with respective pixels of said current block according to said parametric model, and

classifying said pixels of said current block into partitions according to said associated values, wherein a first value according to said parametric model indicates that a pixel of said current block is in said first partition, a second value indicates that a pixel of said current block is in said second partition, and a value in between said first and second values indicates that a pixel of said current block is in a partial surface;

predicting motion vectors for said first and second partitions based on at least one of spatial neighboring blocks and temporal neighboring blocks;

decoding a first motion vector for said first partition and a second motion vector for said second partition based on said predicted motion vectors;

predicting said current block to form a predicted block by:

predicting pixels in said first partition using a first predictor,

predicting pixels in said second partition using a second predictor, and

predicting pixels in said partial surface based on a weighted linear average of said first predictor and said second predictor, wherein said first predictor is weighted by w(x,y) and said second predictor is weighted by (1−w(x,y)) to form said weighted linear average, wherein w(x, y) is said value from said parametric model for pixel at location (x,y); and

decoding said current block based on said predicted block.

16. The method of claim 15 , further comprising:

predicting said parametric model based on another parametric model of one or more spatial neighboring blocks, wherein said parametric model is decoded based on said predicted parametric model.

17. The method of claim 16 , wherein said another parametric model is projected from a spatial neighboring block into said current block to predict said parametric model.

18. A method of video encoding, comprising:

encoding a parametric model for a current block of a picture, wherein said block is partitioned to include a first partition and a second partition according to said parametric model, wherein said partitioning includes:

associating values with respective pixels of said current block according to said parametric model, and

classifying said pixels of said current block into partitions according to said associated values, wherein a first value according to said parametric model indicates that a pixel of said current block is in said first partition, a second value indicates that a pixel of said current block is in said second partition, and a value in between said first and second values indicates that a pixel of said current block is in a partial surface;

predicting motion vectors for said first and second partitions based on at least one of spatial neighboring blocks and temporal neighboring blocks;

encoding a first motion vector for said first partition and a second motion vector for said second partition based on said predicted motion vectors;

predicting said current block to form a predicted block by:

predicting pixels in said first partition using a first predictor,

predicting pixels in said second partition using a second predictor, and

predicting pixels in said partial surface based on a weighted linear average of said first predictor and said second predictor, wherein said first predictor is weighted by w(x,y) and said second predictor is weighted by (1−w(x,y)) to form said weighted linear average, wherein w(x, y) is said value from said parametric model for pixel at location (x,y); and

encoding said current block based on said predicted block.

19. The method of claim 18 , further comprising:

predicting said parametric model based on another parametric model of one or more spatial neighboring blocks, wherein said parametric model is encoded based on said predicted parametric model.

20. The method of claim 19 , wherein said another parametric model is projected from a spatial neighboring block into said current block to predict said parametric model.

21. The method of claim 15 , wherein each of said first and second motion vectors is predicted from an adaptively selected set of motion vectors from spatial and temporal neighboring blocks.

22. The method of claim 18 , wherein each of said first and second motion vectors is predicted from an adaptively selected set of motion vectors from spatial and temporal neighboring blocks.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2024
From: DIVORRA ESCODA, OSCAR; YIN, PENG
To: THOMSON LICENSING
Reel/Frame 066028/0754 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2024
From: THOMSON LICENSING
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 066028/0763 →
Continuity (6)
Continuation 17568311 · Jan 4, 2022
Continuation 17083007 · Oct 28, 2020
Continuation 15482191 · Apr 7, 2017
Division 12309496
Provisional Application 60834993 · Aug 2, 2006
Related Publication 20240129524A1 · Apr 18, 2024
References Cited (36)
US 6097840A · Shiitani et al. · 2000 [cited by applicant]
US 20030059120A1 · Boon et al. · 2003 [cited by applicant]
US 20040091047A1 · Paniconi et al. · 2004 [cited by applicant]
JP H0965338A · 1997 [cited by applicant]
JP H9065338A · 1997 [cited by applicant]
JP 09270015A1 · 1997 [cited by applicant]
JP 2005277968A · 2005 [cited by applicant]
JP 6327003B2 · 2018 [cited by applicant]
JP 8205172B2 · 2018 [cited by applicant]
WO WO2008016605 · 2008 [cited by applicant]
Heising, “Efficient Moving Image Coding Using Grid Based Temporal Prediction,” Section 7 “Modeling of Motion Discontinuities”, Berlin: Technische Universitaet, pp. 1-18, Oct. 2002. [cited by applicant]
Adachi et al., “Refined Results on the Low-Overhead Prediction Modes,” Dec. 4-6, 2001 (Dec. 4-6, 2001), ITU—Telecommunications Standardization Sector, Study Group 16, Question 6, Video Coding Experts Group (VCEG), 15th … [cited by applicant]
Ruhl Uhl G: “Simulation eines Verfahrens zur Bewegungsschatzung und-segmentierung in digitalen Bildsequendzen unter Verwendung eines Blockverzerrungsmodells. PASSAGE” Feb. 1996 (Feb. 1996) Echnische Universtat Berlin, I… [cited by applicant]
Bronshtein et al., “Handbook of mathematics, Passage”, Springer Berlin p. 194 line 15-p. 195, line 15, 2004. [cited by applicant]
Do et al., “On the Compression of Two-Dimensional Piecewise Smooth Functions” Thessaloniki, Greece, 2001 IEEE International Conference on Image Processing (ICIP) 2001. [cited by applicant]
Zhang et al., “Variable Block Size Video Coding with Motion Prediction and Motion Segmentation,” Optomechatronic Micro/Nano Devices and Components III: Oct. 8-10, 2007, Lausanne, Switzerland [Proceeding of SPIE, ISSN 02… [cited by applicant]
Ohm J-R: “Multimedia Communication Technology, 5.2 Signal Enhancement”, Springer, Berlin, De, 2004 p. 181, col. 15-p. 182, col. 19. [cited by applicant]
Cheng, “Visual Pattern Matching in Motion Estimation for Object-Based Very Low Bil-Rate Coding Using Moment-Preserving Edge Detection,” IEEE Transactions on Multimedia, IEEE Service Center, Piscataway, NJ, vol. 7, No. 2… [cited by applicant]
Pandit et al., “On MVC High-Level Syntax for Picture Management”, Joint Video (JVT) of ISO/IEC MPEG ITU-T CEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), JVT-T131, 20th Meeting, pp. 1-6, Jul. 15-21, 2006. [cited by applicant]
Kim et al., “Comments on High-Level Syntax for MVC”, Contribution to the 76th MPEG meeting, ISO/IEC TC1/ FC29/WG1 MPEG2006/ M13319, pp. 1-10, Apr. 2006. [cited by applicant]
Kato et al., “Low-overhead Segment Based Motion Compensation for H.26L”, IEEE, Piscatawy, vol. IV, pp. 3441-3444, NJ, 2002. [cited by applicant]
Martinian et al., “Results of Core Experiment IB on Multiview Coding”, ISO/IEC JTC1/SC29/WG11, Document 13122, pp. 1-6, Apr. 2006. [cited by applicant]
Ohm J-R: “Multimedia Communication Technology, 11 Quantization and Coding”, 2004. Springer, Berlin.DE, p. 445, line 1-p. 450, line 10, p. 458, line 1-line 29, p. 475, line 10-line 34. [cited by applicant]
Schwarz et al., “Tree-structured macroblocked partition” ITU—Telecommunications Standardization Sector Study Group 16 Question 6 Video Coding Experts Group (VCEG), 15th Meeting, Pattaya, Thailand, Dec. 4-6, 2001 (Dec. 4… [cited by applicant]
Hagai et al., “Proposal of Intra Prediction for Improved Macroblock Prediction Modes (VPM2)”, Document JVT-B059, Joint Video Team of 1SO/IEC MPEG and ITU-T VCEG, 2nd Meeting: Geneva, CH, 8 pages, Jan. 29-Feb. 1, 2002. [cited by applicant]
Mueller et al., “Multiview Coding Using AVC”, ISO/IEC JTC1/SC29/WG11, m12945, Coding of Moving Pictures and Associated Audio Information, pp. 1-12, Jan. 2006. [cited by applicant]
Vetro, “Joint Draft 3.0 on Multiview Video Coding”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/ EC JTC1/SC29/WG11 and ITU-T SG16 Q.6, JVT-W209, 23rd Meeting, pp. 1-39, Apr. 21-27, 2007. [cited by applicant]
Donoho, “Wedgelets: Nearly Minimax Estimation of Edges”, The Annalsof statislics, vol. 27, No. 8, Jun. 1999, pp. 859-897. [cited by applicant]
Hung et al., “On Macroblock Partition for Motion Compensation”, Proceedings of The 2006 International Conference on Image Processing (ICIP 2006). vol. 1, 8, IEEE, pp. 1697-1700, Piscataway, NJ, Oct. 2006. [cited by applicant]
Anonymous, “Technologies under Study for Reference Picture Management and High-Level Syntax for Multiview Video Coding”, ISO/IEC JTC1/SC29/WG11, N8018, Coding of Moving Pictures and Audio, pp. 1-10, Apr. 2006. [cited by applicant]
Anonymous, International Telecommunications Union, ITU-T Telecommunication Standardization Sector of ITU,Sries H:Audiovisual Ano Multimedia Systems, Infrastructure of audiovisual services—Coding of Moving Video Advanced… [cited by applicant]
Divorra et al., “Geometry adaptive Block Partitioning”, Video Standards and Drafts, No. VCEG-AF10,Apr. 20, 2007 (Apr. 20, 2007), pp. 1-8, XP030003531 ITU-Telecommunications Standardization Sector, Study group 16 Questio… [cited by applicant]
Kondo et al., “A Motion Compensation Technique Using Sliced Block In Hybrid Video Coding”, In IEEE Intemational Conference on Image Processing 2005, vol. 2, IEEE, pp. 2-6, 2005. [cited by applicant]
Fukuhara T et al: “Very Low Bit-Rate Video Coding with Block Partitioning and Adaptive Selection of Two Time-Differential Frame Memories” Feb. 2007, IEEE Transactions on Circuits and Systems for Video Technology, vol. 7… [cited by applicant]
Kondo et al., “A Motion Compensation Technique Using Sliced Block In Hybrid Video Coding”, IEEE, pp. 1-4, 2005. [cited by applicant]
Kato et al., “Performance Evaluation on H.26L-based Motion Compensation with Segmented Multiple eference Frames” 2002, IEEE, Piscataway, NJ, US. [cited by applicant]