IP Library › Granted Patent US 12,273,540
Granted Patent B2
US 12,273,540 · App. 18/398,903 · Granted Apr 8, 2025

Adaptive partition coding

Inventors: Philipp Merkle (Berlin, DE); Christian Bartnik (Berlin, DE); Haricharan Lakshman (Berlin, DE); Detlev Marpe (Berlin, DE); Karsten Mueller (Berlin, DE); Thomas Wiegand (Berlin, DE); Gerhard Tech (Berlin, DE)
Assignee: DOLBY VIDEO COMPRESSION, LLC
H04N19/176H04N19/103H04N19/105H04N19/119H04N19/157H04N19/196H04N19/23H04N19/44H04N19/46H04N19/593H04N19/597H04N19/70H04N19/82H04N19/96H04N19/14
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,273,540
App. No.
18/398,903
Granted
Apr 8, 2025
Kind
B2
Abstract

Although wedgelet-based partitioning seems to represent a better tradeoff between side information rate on the one hand and achievable variety in partitioning possibilities on the other hand, compared to contour partitioning, the ability to alleviate the constraints of the partitioning to the extent that the partitions have to be wedgelet partitions, enables applying relatively uncomplex statistical analysis onto overlaid spatially sampled texture information in order to derive a good predictor for the bi-segmentation in a depth/disparity map. Thus, in accordance with a first aspect it is exactly the increase of the freedom which alleviates the signaling overhead provided that co-located texture information in form of a picture is present. Another aspect pertains the possibility to save side information rate involved with signaling a respective coding mode supporting irregular partitioning.

Claims (41)

1. A decoder for reconstructing a block of a depth map associated with a texture picture from a data stream, configured to:

determine a texture threshold based on an operation performed on sample values of a reference block of a texture picture,

determine a partitioning of the reference block of the texture picture based on the texture threshold;

obtain a partition of a block of a depth map such that the block of the depth map is segmented into two portions based on the partitioning of the reference block of the texture picture; and

reconstruct the block of the depth map by decoding the two portions associated with the partition of the block of the depth map.

2. The decoder according to claim 1 , further configured to:

determine the partitioning of the reference block of the texture picture based on a comparison of one or more sample values of the reference block with the texture threshold; and

use a coding option identifier from the data stream as an index to perform the determining of the texture threshold, the determining of the partitioning of the reference block, and the transferring the partitioning.

3. The decoder according to claim 2 , configured to, in determining the partitioning of the reference block, individually check values of the texture picture within the reference block at tiles of a two-dimensional subdivision of the reference block, as to whether the respective value is greater than or lower than a respective predetermined value, so that each of first and second portions of the reference block of the texture picture is a set of the tiles which together completely cover the reference block of the texture picture and are complementary to each other.

4. The decoder according to claim 3 , configured to, in determining the partitioning of the reference block, individually check values of the texture picture within the reference block at sample resolution so that each tile corresponds to a sample position of the reference block.

5. The decoder according to claim 2 , configured to, in determining the partitioning of the reference block, apply morphological hole filling and/or low-pass filtering onto a result of the comparing in order to acquire two portions of the reference block.

6. The decoder according to claim 2 , configured to, in determining the texture threshold, determine a measure for a central tendency of reconstructed sample values of the reference block of the texture picture, wherein the measure represents the texture threshold.

7. The decoder according to claim 2 , wherein the partitioning of the reference block is determined by comparing each sample of the reference block with the texture threshold.

8. An encoder for encoding a block of a depth map associated with a texture picture into a data stream, configured to:

determine a texture threshold based on an operation performed on sample values of a reference block of a texture picture,

determine a partitioning of the reference block of the texture picture based on the texture threshold;

obtain a partition of a block of a depth map such that the block of the depth map is segmented into two portions based on the partitioning of the reference block of the text texture picture; and

encode, into the data stream, the block of the depth map based on the two portions associated with the partition of the block of the depth map.

9. The encoder according to claim 8 , further configured to:

determine the partitioning of the reference block of the texture picture based on a comparison of one or more sample values of the reference block with the texture threshold; and

use a coding option identifier from the data stream as an index to perform the determining of the texture threshold, the determining of the partitioning of the reference block, and the transferring the partitioning.

10. The encoder according to claim 9 , configured to, in determining the partitioning of the reference block, individually check values of the texture picture within the reference block at tiles of a two-dimensional subdivision of the reference block, as to whether the respective value is greater than or lower than a respective predetermined value, so that each of first and second portions of the reference block of the texture picture is a set of the tiles which together completely cover the reference block of the texture picture and are complementary to each other.

11. The encoder according to claim 10 , configured to, in determining the partitioning of the reference block, individually check values of the texture picture within the reference block at sample resolution so that each tile corresponds to a sample position of the reference block.

12. The encoder according to claim 9 , configured to, in determining the partitioning of the reference block, apply morphological hole filling and/or low-pass filtering onto a result of the comparing in order to acquire two portions of the reference block.

13. The encoder according to claim 9 , configured to, in determining the texture threshold, determine a measure for a central tendency of reconstructed sample values of the reference block of the texture picture, wherein the measure represents the texture threshold.

14. The encoder according to claim 9 , wherein the partitioning of the reference block is determined by comparing each sample of the reference block with the texture threshold.

15. A non-transitory computer-readable medium for storing data associated with a video, comprising:

a data stream stored in the non-transitory computer-readable medium, the data stream comprising encoded information of a reference block of a texture picture of the video and encoded information of a block in a depth map of the video, wherein the block in the depth map is decoded using a plurality of operations including:

determining a texture threshold based on an operation performed on sample values of a reference block of a texture picture,

determining a partitioning of the reference block of the texture picture based on the texture threshold;

obtaining a partition of a block of a depth map such that the block of the depth map is segmented into two portions based on the partitioning of the reference block of the texture picture; and

reconstructing the block of the depth map by decoding the two portions associated with the partition of the block in the depth map.

16. The computer-readable medium according to claim 15 , wherein the plurality of operations further comprises:

determining the partitioning of the reference block of the texture picture based on a comparison of one or more sample values of the reference block with the texture threshold; and

using a coding option identifier from the data stream as an index to perform the determining of the texture threshold, the determining of the partitioning of the reference block, and the transferring the partitioning.

17. The computer-readable medium according to claim 16 , wherein determining the partitioning of the reference block includes individually checking values of the texture picture within the reference block at tiles of a two-dimensional subdivision of the reference block, as to whether the respective value is greater than or lower than a respective predetermined value, so that each of first and second portions of the reference block of the texture picture is a set of the tiles which together completely cover the reference block of the texture picture and are complementary to each other.

18. The computer-readable medium according to claim 17 , wherein determining the partitioning of the reference block includes:

individually checking values of the texture picture within the reference block at sample resolution so that each tile corresponds to a sample position of the reference block; and/or

applying morphological hole filling and/or low-pass filtering onto a result of the comparing in order to acquire two portions of the reference block.

19. The computer-readable medium according to claim 16 , wherein determining the texture threshold includes determining a measure for a central tendency of reconstructed sample values of the reference block of the texture picture, wherein the measure represents the texture threshold.

20. The computer-readable medium according to claim 16 , wherein the partitioning of the reference block is determined by comparing each sample of the reference block with the texture threshold.

Assignments (2)
CHANGE OF NAME Recorded Jan 30, 2026
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 074536/0781 →
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0772 →
Continuity (9)
Continuation 17170289 · Feb 8, 2021
Continuation 17012506 · Sep 4, 2020
Continuation 16720054 · Dec 19, 2019
Continuation 16437185 · Jun 11, 2019
Continuation 15663256 · Jul 28, 2017
Continuation 14273601 · May 9, 2014
Continuation PCTEP2012072328 · Nov 9, 2012
Provisional Application 61558631 · Nov 11, 2011
Related Publication 20240137531A1 · Apr 25, 2024
References Cited (99)
US 6243416B1 · Matsushiro et al. · 2001 [cited by applicant]
US 6404817B1 · Saha et al. · 2002 [cited by applicant]
US 6944351B2 · Kim et al. · 2005 [cited by applicant]
US 7596279B2 · Sugimoto et al. · 2009 [cited by applicant]
US 7756348B2 · Mukherjee et al. · 2010 [cited by applicant]
US 8249273B2 · Inoda et al. · 2012 [cited by applicant]
US 8666177B2 · Chen et al. · 2014 [cited by applicant]
US 10574982B2 · Merkle et al. · 2020 [cited by applicant]
US 20020094125A1 · Guo · 2002 [cited by applicant]
US 20040101059A1 · Joch et al. · 2004 [cited by applicant]
US 20050038837A1 · Marpe et al. · 2005 [cited by applicant]
US 20050105615A1 · El-Maleh · 2005 [cited by applicant]
US 20060078051A1 · Liang et al. · 2006 [cited by applicant]
US 20080101707A1 · Mukherjee et al. · 2008 [cited by applicant]
US 20090103615A1 · Francois et al. · 2009 [cited by applicant]
US 20090214133A1 · Aoyama · 2009 [cited by applicant]
US 20090225834A1 · Song et al. · 2009 [cited by applicant]
US 20100208827A1 · Divorra Escoda et al. · 2010 [cited by applicant]
US 20110164677A1 · Lu et al. · 2011 [cited by applicant]
US 20110206132A1 · Bivolarsky et al. · 2011 [cited by applicant]
US 20110274166A1 · Jeon · 2011 [cited by applicant]
US 20110274176A1 · Panusopone et al. · 2011 [cited by applicant]
US 20120008684A1 · Lee et al. · 2012 [cited by applicant]
US 20120200669A1 · Lai · 2012 [cited by examiner]
US 20130027230A1 · Marpe · 2013 [cited by applicant]
US 20130034171A1 · Winken · 2013 [cited by examiner]
US 20130147915A1 · Wiegand et al. · 2013 [cited by applicant]
US 20140079132A1 · Amon · 2014 [cited by applicant]
CN 1777289A · 2006 [cited by applicant]
CN 101502120A · 2009 [cited by applicant]
CN 101536528A · 2009 [cited by applicant]
CN 101609548A · 2009 [cited by applicant]
CN 101731012A · 2010 [cited by applicant]
CN 101969566A · 2011 [cited by applicant]
CN 101453684B · 2014 [cited by applicant]
EP 1924075A2 · 2008 [cited by applicant]
EP 2052546A1 · 2009 [cited by applicant]
EP 2521357A1 · 2012 [cited by applicant]
EP 2860977A1 · 2015 [cited by applicant]
EP 2777285B1 · 2017 [cited by applicant]
EP 8349464A1 · 2018 [cited by applicant]
JP H01303888A · 1989 [cited by applicant]
JP H06153167A · 1994 [cited by applicant]
JP 0799581A · 1995 [cited by applicant]
JP H07099581A · 1996 [cited by applicant]
JP H08298665A · 1996 [cited by applicant]
JP H09275565A · 1997 [cited by applicant]
JP 2004072732A · 2004 [cited by applicant]
JP 2008516561A · 2008 [cited by applicant]
JP 2009017502A · 2009 [cited by applicant]
JP 2009147968A · 2009 [cited by applicant]
JP 2009206713A · 2009 [cited by applicant]
JP 2019051540A · 2019 [cited by applicant]
JP 6788699A · 2020 [cited by applicant]
JP 5814783B2 · 2021 [cited by applicant]
KR 1020110117075A · 2011 [cited by applicant]
WO 2006033953A1 · 2006 [cited by applicant]
WO 2008007913A1 · 2008 [cited by applicant]
WO 2010082463A1 · 2010 [cited by applicant]
WO WO2011046607A2 · 2011 [cited by examiner]
WO 2011066672A1 · 2011 [cited by applicant]
WO 2011074896A2 · 2011 [cited by applicant]
WO 2011128365A1 · 2011 [cited by applicant]
WO WO2011127966A1 · 2011 [cited by examiner]
WO 2012020092A1 · 2012 [cited by applicant]
WO 2012147740A1 · 2012 [cited by applicant]
WO 2013032423A1 · 2013 [cited by applicant]
Housi Bi, et al. (Editors), “New Generation Video Compression Coding Standard—H.264/VC” (Second Edition), Nov. 30, 2009. [cited by applicant]
Christian Bartnik et al., HEVC Extension for Multiview Video Coding and Multiview Video plus Depth Coding, ITU—Telecommunications Standardization Sector Study Group 16 Question 6 Video Coding Experts Group, VCEG-AR13, 4… [cited by applicant]
Min-Koo Kang, SPIE Visual Information Processing and Communication, Geometry-based partitioning for efficient intraprediction in depth video coding, 2020. [cited by applicant]
Davies, T. et al., “Suggestion for a Test Model”, ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 1st Meeting, Document JC1VC-A033, 30 pages, Apr. 15-23, 2010. [cited by applicant]
Escoda, 0. et al., “Geometry-Adaptive Block Partitioning for Video Coding”, Proc. IEEE ICASSP, 2007, pp. 1-657-1-660. [cited by applicant]
Gao, J. et al., “Application of Wedgelet Transform for Image Process”, Proc. IEEE ICACIA, pp. 220-222, 2008. [cited by applicant]
ITU-T Recommendation H.264, Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving Video, ISO/IEC JTC 1, 14496-10, 732 pages, Apr. 2013. [cited by applicant]
Kang, M. et al., “Adaptive Geometry-based Intra Prediction for Depth Video Coding”, IEEE International Conference on Multimedia and Expo, Jul. 19, 2010, pp. 1230-1235. [cited by applicant]
Loke. K., “Wedgelets-Based Automatic Object Contour Detection”. Proc. IEEE ICNC. pp. 3664-3668. 2010. [cited by applicant]
Liu, S. et al., “New Depth Coding Techniques with Utilization of Corresponding Video” IEEE Transactions on Broadcasting, vol. 57, No. 2, Jun. 2011, pp. 551-561. [cited by applicant]
Liu, S. et al., “Sparse Dyadic Mode for Depth Map Compression”, Proc. IEEE ICIP, Sep. 26-29, 2010, pp. 3421-3424. [cited by applicant]
Merkle, P. et al., “3D-CE6.H Related: Fast Wedgelet Search”, ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 1st Meeting, Document JCT2-A0105, 4 pages, Jul. 16-20, 2012. [cited by applicant]
Merkle, P. et al., “The Effects of Multiview Depth Video Compression on Multiview Rendering”, Signal Processing: Image Communication, 24, 2009, pp. 73-88. [cited by applicant]
Merkle, P., “CE6.H Results on Simplified Wedgelet Search for DMM Modes 1 and 3”, ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 2nd Meeting, Document JCT3V-B0039, 7 pages, 2012. [cited by applicant]
Morvan, Y et al., “Novel Coding Technique for Depth Images using Quadtree Decomposition and Plane Approximation”, Proc. SPIE, vol. 5960,2005, pp. 1187-1194. [cited by applicant]
Ostermann, J., “Coding of Binary Shape in MPEG-4”, ITG Fachberichte, pp. 659-662, 1997. [cited by applicant]
Smolic, A. et al., “Development of a New MPEG Standard for Advanced 3D Video Applications”, Proc. IEEE ISPA, pp. 400-407, 2009. [cited by applicant]
Wakin, M. et al., “Rate-Distortion Optimized Image Compression Using Wedgelets”, Proc. IEEE ICIP, pp. 111-237-111-240, 2002. [cited by applicant]
Yang, W. et al., “An MPEG-4-Compatible Stereoscopic/Multiview Video Coding Scheme”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, No. 2, pp. 286-290, 2006. [cited by applicant]
Zhang, Y. et al., “Motion Compensation Using Polyline Based Block Partition”, 2nd International Congress on Image and Signal Processing, Oct. 17-19, 2009, 5 pages. [cited by applicant]
Ferreira et al., “Efficiency improvements for geometric-partition-based video coder”, Proc. IEEE Int. Conf. Image Process., Cairo Egyps, Nov. 2009, pp. 1009-1012. [cited by applicant]
Gerhard Tech et al., “3D-HEVC Test Model 1”, Joint Collaborative Team on 3D Video Coding Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 1st Meeting: Stockholm, SE, Jul. 16-20, 2012, 83 pgs. [cited by applicant]
Prosecution History of U.S. Pat. No. 10,574,982. [cited by applicant]
Declaration of Dr. Immanuel Freedman, including Appendix A (CV of Dr. Freedman), Jul. 2, 2024. [cited by applicant]
“On Macroblock Partition for Motion Compensation,” Proceedings of the International Conference on Image Processing, Nov. 2006, Hung, et al. [cited by applicant]
“Masking”, School of Informatics, WebArchive, Sep. 9, 2004, Fisher et al., https://web.archive.org/web/20040909175026 https:/homepages.inf.ed.ac.uk/rbf/HIPR2/mask.htm. [cited by applicant]
“EDM: Electron Direct Methods,” WebArchive, Dec. 15, 2004 (“EDM”) https://web.archive.org/web/20041215233117 http://www.nurnis.northwestem.edu/edrn/documentation/edm.htm. [cited by applicant]
“Binary Image Analysis,” Computer Vision, Mar. 2000, Shapiro et al. [cited by applicant]
“Multiscale Wedgelet Image Analysis: Fast Decompositions and Modeling”, IEEE, 2002, Romberg et al. [cited by applicant]
“Segmentation and Thresholding” CRC Press LLC, 2002. [cited by applicant]
“A Survey of Various Data Compression Techniques,” Apr. 1, 2010, Smith. [cited by applicant]
Min-Koo Kang, et al., “Geometry-based Block Partitioning for Efficient Intra Prediction in Depth Video Coding”, Visual Information Processing and Communication, vol. 7543, 2010. [cited by applicant]