IP Library Granted Patent US 11,102,518
Granted Patent B2
US 11,102,518 · App. 16/855,266 · Granted Aug 24, 2021

Coding of a spatial sampling of a two-dimensional information signal using sub-division

Inventors: Heiner Kirchhoffer (Berlin, DE); Martin Winken (Berlin, DE); Philipp Helle (Berlin, DE); Detlev Marpe (Berlin, DE); Heiko Schwarz (Panketal, DE); Thomas Wiegand (Berlin, DE)
Assignee: GE Video Compression, LLC
H04N19/96H04N19/17H04N19/176H04N19/1883H04N19/46H04N19/52H04N19/53H04N19/59H04N19/593H04N19/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,102,518
App. No.
16/855,266
Granted
Aug 24, 2021
Kind
B2
Abstract

Coding schemes for coding a spatially sampled information signal using sub-division and coding schemes for coding a sub-division or a multitree structure are described, wherein representative embodiments relate to picture and/or video coding applications.

Claims (68)

1. A decoder comprising:

an extractor configured to:

extract, from a data stream representing a video, first subdivision flags and second subdivision flags, wherein the first subdivision flags are associated with prediction coding, each of the first subdivision flags is associated with one of a first set of regions in a multi-tree structure, the second subdivision flags are associated with transform coding, and each of the second subdivision flags is associated with one of a second set of regions in the multi-tree structure, entropy decode each of the first subdivision flags using a first probability estimation context, which is determined based on a hierarchy level within the multi-tree structure of a region in the first set of regions associated with the first subdivision flag, and

entropy decode each of the second subdivision flags using a second probability estimation context, which is determined based on a size of a region in the second set of regions associated with the second subdivision flag;

a divider configured to:

divide, based on one or more of the first subdivision flags, an array of information samples representing a spatially sampled portion of the video into the first set of regions of the multi-tree structure, and

divide, based on one or more of the second subdivision flags, at least a portion of the array of information samples into the second set of regions of the multi-tree structure; and a reconstructor configured to reconstruct the array of information samples using prediction coding in accordance with the first set of regions and transform coding in accordance with the second set of regions.

2. The decoder according to claim 1 , wherein the first probability estimation context is the same for all first regions that are associated with the same hierarchy level and is different for first regions associated with a different hierarchy level.

3. The decoder according to claim 1 , wherein the second probability estimation context is the same for all second regions that are of the same size and is different for second regions of a different size.

4. The decoder according to claim 1 , wherein the divider is configured to perform the division of the array of information samples into the first set of regions such that the first set of regions are rectangular blocks of a size determined by a first maximum region size, wherein the rectangular blocks are arranged in an order to gaplessly cover the array of information samples.

5. The decoder according to claim 1 , wherein the divider is configured to subdivide at least some of the first set of regions by:

determining whether a respective first subdivision flag indicates that each of the at least some of the first set of regions is to be sub-divided,

based on a determination that each of the at least some of the first set of regions is to be subdivided, sub-dividing that region into sub-regions of a first hierarchy level within the multi-tree structure according to a partition rule associated with the first hierarchy level, and

recursively repeating the determination and sub-division for the sub-regions of the first hierarchy level in order to obtain regions of higher-order hierarchy levels using respective partition rules associated therewith,

wherein the determination and sub-division is performed until no further division is to be performed according to the first subdivision flags or a maximum hierarchy level is reached.

6. The decoder according to claim 5 , wherein the divider is configured to, in accordance with the partition rules associated with the first and higher-order hierarchy levels, sub-divide into sub-regions of equal size, such that there are an equal number of sub-regions at each hierarchy level.

7. The decoder according to claim 1 , wherein the extractor is configured to extract syntax elements associated with the first or second set of regions from the data stream in a depth-first traversal order.

8. The decoder according to claim 1 , further comprising: a merger configured to combine, based on a first subset of syntax elements of the data stream, disjoint from a second subset of syntax elements of the data stream including the first subdivision flags, spatially neighboring ones of the first set of regions to obtain an intermediate subdivision of the array of information samples, wherein the reconstructor is configured to reconstruct the array of information samples using the intermediate subdivision.

9. The decoder according to claim 1 , further comprising a predictor configured to predict the array of information samples from the data stream, wherein the reconstructor is configured to perform a spectral-to-spatial domain transform coding to obtain a residual for the array of information samples, and combine the residual and the prediction of the array of information samples to reconstruct the array of information samples.

10. The decoder according to claim 1 , wherein the recursive multi-tree partitioning includes a quadtree partitioning technique.

11. A method comprising:

extracting, from a data stream representing a video, first subdivision flags and second subdivision flags, wherein the first subdivision flags are associated with prediction coding, each of the first subdivision flags is associated with one of a first set of regions in a multi-tree structure, the second subdivision flags are associated with transform coding, and each of the second subdivision flags is associated with one of a second set of regions in the multi-tree structure;

entropy decoding each of the first subdivision flags using a first probability estimation context, which is determined based on a hierarchy level within the multi-tree structure of a region in the first set of regions associated with the first subdivision flag;

entropy decoding each of the second subdivision flags using a second probability estimation context, which is determined based on a size of a region in the second set of regions associated with the second subdivision flag;

dividing, based on one or more of the first subdivision flags, an array of information samples representing a spatially sampled portion of the video into the first set of regions of the multi-tree structure;

dividing, based on one or more of the second subdivision flags, at least a portion of the array of information samples into the second set of regions of the multi-tree structure; and

reconstructing the array of information samples using prediction coding in accordance with the first set of regions and transform coding in accordance with the second set of regions.

12. The method according to claim 11 , wherein the first probability estimation context is the same for all first regions that are associated with the same hierarchy level and is different for first regions associated with a different hierarchy level.

13. The method according to claim 11 , wherein the second probability estimation context is the same for all second regions that are of the same size and is different for second regions of a different size.

14. The method according to claim 11 , wherein the dividing the array of information samples into the first set of regions such that the first set of regions are rectangular blocks of a size determined by a first maximum region size, wherein the rectangular blocks are arranged in an order to gaplessly cover the array of information samples.

15. The method according to claim 11 , further comprising sub-dividing at least some of the first set of regions including:

determining whether a respective first subdivision flag indicates that each of the at least some of the first set of regions is to be sub-divided,

based on a determination that each of the at least some of the first set of regions is to be subdivided, sub-dividing that region into sub-regions of a first hierarchy level within the multi-tree structure according to a partition rule associated with the first hierarchy level, and

recursively repeating the determination and sub-division for the sub-regions of the first hierarchy level in order to obtain regions of higher-order hierarchy levels using respective partition rules associated therewith, wherein the determination and sub-division is performed until no further division is to be performed according to the first subdivision flags or a maximum hierarchy level is reached.

16. The method according to claim 15 , wherein, in accordance with the partition rules associated with the first and higher-order hierarchy levels, the sub-dividing is performed to obtain sub-regions of equal size, such that there are an equal number of sub-regions at each hierarchy level.

17. The method according to claim 11 , further comprising extracting syntax elements associated with the first or second set of regions from the data stream in a depth-first traversal order.

18. The method according to claim 11 , further comprising:

predicting the array of information samples from the data stream, wherein the reconstructing includes performing a spectral-to-spatial domain transform coding to obtain a residual for the array of information samples, and combining the residual and the prediction of the array of information samples to reconstruct the array of information samples.

19. The method according to claim 11 , wherein the recursive multi-tree partitioning includes a quadtree partitioning technique.

20. An encoder comprising:

a divider configured to:

divide, based on one or more of first subdivision flags, an array of information samples representing a spatially sampled portion of a video into a first set of regions of a multi-tree structure, and

divide, based on one or more of second subdivision flags, at least a portion of the array of information samples into a second set of regions of the multi-tree structure; and a data stream generator configured to:

encode the array of information samples using prediction coding in accordance with the first set of regions and transform coding in accordance with the second set of regions,

entropy encode each of the first subdivision flags using a first probability estimation context, which is determined based on a hierarchy level within a multi-tree structure of a region in the first set of regions associated with the first subdivision flag,

entropy encode each of the second subdivision flags using a second probability estimation context, which is determined based on a size of a region in the second set of regions associated with the second subdivision flag, and

insert into a data stream the encoded array of information samples, the entropy encoded first subdivision flags and the entropy encoded second subdivision flags.

21. The encoder according to claim 20 , wherein the first probability estimation context is the same for all first regions that are associated with the same hierarchy level and is different for first regions associated with a different hierarchy level.

22. The encoder according to claim 20 , wherein the second probability estimation context is the same for all second regions that are of the same size and is different for second regions of a different size.

23. The encoder according to claim 20 , wherein the divider is configured to perform the division of the array of information samples into the first set of regions such that the first set of regions are rectangular blocks of a size determined by a first maximum region size, wherein the rectangular blocks are arranged in an order to gaplessly cover the array of information samples.

24. The encoder according to claim 20 , wherein the divider is configured to subdivide at least some of the first set of regions by:

determining whether a respective first subdivision flag indicates that each of the at least some of the first set of regions is to be sub-divided,

based on a determination that each of the at least some of the first set of regions is to be subdivided, sub-dividing that region into sub-regions of a first hierarchy level according to a partition rule associated with the first hierarchy level, and

recursively repeating the determination and sub-division for the sub-regions of the first hierarchy level in order to obtain regions of higher-order hierarchy levels using respective partition rules associated therewith,

wherein the determination and sub-division is performed until no further division is to be performed according to the first subdivision information or a maximum hierarchy level is reached.

25. The encoder according to claim 24 , wherein the divider is configured to, in accordance with the partition rules associated with the first and higher-order hierarchy levels, sub-divide into sub-regions of equal size, such that there are an equal number of sub-regions at each hierarchy level.

26. A non-transitory computer-readable medium for storing data associated with a video, comprising:

a data stream stored in the non-transitory computer-readable medium, the data stream comprising encoded information associated with an array of information samples representing a spatially sampled portion of the video, the encoded information including first subdivision flags and second subdivision flags, wherein the first subdivision flags are associated with prediction coding, each first subdivision flag is associated with one of a first set of regions of a multi-tree structure, the second subdivision flags are associated with transform coding, and each second subdivision flag is associated with one of a second set of regions of the multi-tree structure, the encoded information is coded into the data stream by operations including:

dividing, based on one or more of first subdivision flags, an array of information samples representing a spatially sampled portion of a video into a first set of regions of a multi-tree structure;

dividing, based on one or more of second subdivision flags, at least a portion of the array of information samples into a second set of regions of the multi-tree structure;

encoding the array of information samples using prediction coding in accordance with the first set of regions and transform coding in accordance with the second set of regions;

entropy encoding each of the first subdivision flags using a first probability estimation context, which is determined based on a hierarchy level within a multi-tree structure of a region in the first set of regions associated with the first subdivision flag;

entropy encoding each of the second subdivision flags using a second probability estimation context, which is determined based on a size of a region in the second set of regions associated with the second subdivision flag; and

inserting into a data stream the encoded array of information samples, the entropy encoded first subdivision flags and the entropy encoded second subdivision flags.

27. The computer-readable medium according to claim 26 , wherein the first probability estimation context is the same for all first regions that are associated with the same hierarchy level and is different for first regions associated with a different hierarchy level.

28. The computer-readable medium according to claim 26 , wherein the second probability estimation context is the same for all second regions that are of the same size and is different for second regions of a different size.

29. The computer-readable medium according to claim 26 , wherein syntax elements associated with the first or second set of regions are inserted into the data stream in a depth-first traversal order.

30. The computer-readable medium according to claim 26 , wherein the recursive multi-tree partitioning includes a quadtree partitioning technique.

Assignments (3)
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0446 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2020
From: KIRCHHOFFER, HEINER; WINKEN, MARTIN; HELLE, PHILIPP; MARPE, DETLEV; SCHWARZ, HEIKO; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V
Reel/Frame 052464/0511 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2020
From: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V
To: GE VIDEO COMPRESSION, LLC
Reel/Frame 052467/0248 →
Priority Claims (2)
EP 10159819 · Apr 13, 2010 · regional
WO PCT/EP2010/054843 · Apr 13, 2010 · international
Continuity (7)
Continuation 16561427 · Sep 5, 2019
Continuation 16155281 · Oct 9, 2018
Continuation 15413852 · Jan 24, 2017
Continuation 15195407 · Jun 28, 2016
Continuation 13649251 · Oct 11, 2012
Continuation PCTEP2011055534 · Apr 8, 2011
Related Publication 20200252658A1 · Aug 6, 2020
Cited By (2)
US 12,513,307 US 12,561,715