IP Library Granted Patent US 11,983,737
Granted Patent B2
US 11,983,737 · App. 18/052,520 · Granted May 14, 2024

Region merging and coding parameter reuse via merging

Inventors: Philipp Helle (Berlin, DE); Simon Oudin (Berlin, DE); Martin Winken (Berlin, DE); Detlev Marpe (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: GE Video Compression, LLC
G06Q30/0251G06F16/00G06Q30/02G06Q30/0601H04N19/105H04N19/119H04N19/139H04N19/14H04N19/147H04N19/176H04N19/186H04N19/196H04N19/197H04N19/46H04N19/463H04N19/52H04N19/61H04N19/96G06Q10/00G06Q30/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,983,737
App. No.
18/052,520
Granted
May 14, 2024
Kind
B2
Abstract

A favorable merging or grouping of simply connected regions into which the array of information samples is sub-divided, is coded with a reduced amount of data. To this end, a predetermined relative locational relationship is defined enabling an identifying, for a predetermined simply connected region, of simply connected regions within the plurality of simply connected regions which have the predetermined relative locational relationship to the predetermined simply connected region. Namely, if the number is zero, a merge indicator for the predetermined simply connected region may be absent within the data stream. In other embodiments, spatial sub-division is performed depending on a first subset of syntax elements, followed by combining spatially neighboring simply connected regions depending on a second subset of syntax elements, to obtain an intermediate sub-division.

Claims (70)

1. A decoder, comprising:

an extractor configured to extract payload data from a data stream for each of a plurality of blocks into which an array of information samples representing spatially sampled information of a color plane of a picture is partitioned, the picture containing multiple color planes; and

a reconstructor configured to reconstruct the picture from the payload data for each block of the array of information samples, by processing, for each block, the payload data in a way prescribed by coding parameters associated with the respective block,

wherein the extractor is further configured to:

for a predetermined block, extract combined merge syntax elements for the predetermined block from the data stream, each of the combined merge syntax elements indicating whether merge processing of the predetermined block is to be applied and if so, identifying, for the respective combined merge syntax element, a certain block of two possible blocks for merge processing, wherein the certain block has a predetermined relative locational relationship to the predetermined block,

if one or more of the combined merge syntax elements merge processing of the predetermined block, adopt the coding parameters of the certain block as the coding parameters for the predetermined block, for the multiple color planes of the picture, and if merge processing is not indicated by the combined merge syntax elements, extract, for each of the multiple color planes of the picture, coding parameters for the predetermined block from the data stream.

2. A decoder for decoding a data stream into which a picture is coded, comprising:

a sub-divider configured to spatially sub-divide, depending on a first subset of syntax elements contained in the data stream, an array of information samples representing a spatially sampling of the picture into a plurality of simply connected regions of different sizes by recursively multi-partitioning;

a merger configured to merge, depending on a second subset of syntax elements within the data stream, being disjoined from the first subset, predetermined simply connected regions with spatially neighboring simply connected regions, wherein the second subset of syntax elements includes combined merge syntax elements, each indicating whether merge processing of a predetermined simply connected region is to be applied and if so, identifying, for the respective combined merge syntax element, a certain spatially neighboring simply connected region of two possible spatially neighboring simply connected regions for merge processing, wherein the certain spatially neighboring simply connected region has a predetermined relative locational relationship to the predetermined simply connected region; and

a reconstructor configured to reconstruct the array of information samples from the data stream with adopting or predicting coding parameters for the predetermined simply connected regions from the certain neighboring simply connected regions with which the predetermined simply connected regions are merged for multiple color planes of the picture and extracting, for each of the multiple color planes, the coding parameters of simply connected regions which are not merged from the data stream.

3. The decoder according to claim 1 , wherein the data stream comprises depth information.

4. The decoder according to claim 1 , wherein the data stream has a sequence of pictures with each picture comprising an array of luma samples along with two arrays of chroma samples, wherein a scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a scaling factor for a spatial resolution in the vertical direction.

5. The decoder according to claim 1 , wherein the decoder is configured to decode the multiple color planes of the picture independently.

6. A method for decoding, comprising:

extracting payload data from a data stream for each of a plurality of blocks into which an array of information samples representing spatially sampled information of a color plane of a picture is partitioned, the picture containing multiple color planes; and

reconstructing the picture from the payload data for each block of the array of information samples, by processing, for each block, the payload data in a way prescribed by coding parameters associated with the respective block,

wherein the extracting comprises:

for a predetermined block, extracting combined merge syntax elements for the predetermined block from the data stream, each of the combined merge syntax elements indicating whether merge processing of the predetermined block is to be applied and if so, identifying, for the respective combined merge syntax element, a certain block of two possible blocks for merge processing, wherein the certain block has a predetermined relative locational relationship to the predetermined block,

if one or more of the combined merge syntax elements suggest merge processing of the predetermined block, adopting the coding parameters of the certain block as the coding parameters for the predetermined block, for the multiple color planes of the picture, and if merge processing is not indicated by the combined merge syntax elements, extracting, for each of the multiple color planes of the picture, coding parameters for the predetermined block from the data stream.

7. A method for decoding a data stream into which a picture is coded, comprising:

spatially sub-dividing, depending on a first subset of syntax elements contained in the data stream, an array of information samples representing a spatially sampling of the picture into a plurality of simply connected regions of different sizes by recursively multi-partitioning;

merging, depending on a second subset of syntax elements within the data stream, being disjoined from the first subset, predetermined simply connected regions with spatially neighboring simply connected regions, wherein the second subset of syntax elements includes combined merge syntax elements, each indicating whether merge processing of a predetermined simply connected region is to be applied and if so, identifying, for the respective combined merge syntax element, a certain spatially neighboring simply connected region of two possible spatially neighboring simply connected regions for merge processing, wherein the certain spatially neighboring simply connected region has a predetermined relative locational relationship to the predetermined simply connected region; and

reconstructing the array of information samples from the data stream with adopting or predicting coding parameters for the predetermined simply connected regions from the certain neighboring simply connected regions with which the predetermined simply connected regions are merged for multiple color planes of the picture and extracting, for each of the multiple color planes, the coding parameters of simply connected regions which are not merged from the data stream.

8. The method according to claim 6 , the data stream comprises depth information.

9. The method according to claim 6 , wherein the data stream has encoded thereinto a sequence of pictures with each picture comprising an array of luma samples along with two arrays of chroma samples, wherein a scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a scaling factor for a spatial resolution in the vertical direction.

10. The method according to claim 6 , wherein the multiple color planes of the picture are decoded independently.

11. An encoder configured to encode an array of information samples representing spatially sampled information of a color plane of a picture into payload data for each of a plurality of blocks into which the array of information samples is sub-divided, and coding parameters associated with the respective block so as to prescribe the way the picture is to be reconstructed from the payload data for the respective block,

wherein the encoder is further configured to:

for a predetermined block, insert combined merge syntax elements for the predetermined block into a data stream, each of the combined merge syntax elements indicating whether merge processing of the predetermined block is to be applied and if so, identifies, for the respective combined merge syntax element, a certain block of two possible blocks, wherein the certain block has a predetermined relative locational relationship to the predetermined block, wherein if one or more of the combined merge syntax elements suggest merge processing of the predetermined block, the coding parameters of the certain block are to be adopted as the coding parameters for the predetermined block, for multiple color planes of the picture, and, if merge processing is not indicated by the combined merge syntax elements, insert, for each of the multiple color planes of the picture, coding parameters for the predetermined block into the data stream.

12. An encoder for generating a data stream into which a picture is coded, comprising:

a subdivision/merge stage configured to determine a first subset of syntax elements defining a spatial sub-division of an array of information samples representing a spatially sampling of the picture into a plurality of simply connected regions of different sizes by recursively multi-partitioning, and a second subset of syntax elements being disjoined from the first subset, defining a merging of predetermined simply connected regions with spatially neighboring simply connected regions, wherein the second subset of syntax elements includes combined merge syntax elements, each indicating whether merge processing of a predetermined simply connected region is to be applied and if so, identifying, for the respective combined merge syntax element, a certain spatially neighboring simply connected region of two spatially neighboring simply connected regions for merge processing, wherein the certain spatially neighboring simply connected region has a predetermined relative locational relationship to the predetermined simply connected region; and

a data stream generator configured to code the array of information samples into the data stream with inserting the first and second subsets of syntax elements into the data stream,

wherein coding parameters for the predetermined simply connected regions are indicated by way of the second subset of syntax elements as to be adopted or predicted from the certain neighboring simply connected regions with which the predetermined simply connected regions are merged for multiple color planes of the picture,

wherein a data stream inserter is configured to insert, for each of the multiple color planes, the coding parameters of simply connected regions which are not merged into the data stream.

13. The encoder according to claim 11 , wherein the data stream comprises depth information.

14. The encoder according to claim 11 , wherein the data stream has encoded thereinto a sequence of pictures with each picture comprising an array of luma samples along with two arrays of chroma samples,

wherein a scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a scaling factor for a spatial resolution in the vertical direction.

15. The encoder according to claim 11 , wherein the encoder is configured to encode the multiple color planes of the picture independently.

16. A method for encoding an array of information samples representing spatially sampled information of a color plane of a picture into payload data for each of a plurality of blocks into which the array of information samples is sub-divided, and coding parameters associated with the respective block so as to prescribe the way the picture is to be reconstructed from the payload data for the respective block,

wherein the method comprising:

for a predetermined block, inserting combined merge syntax elements for the predetermined block into a data stream, each of the combined merge syntax elements indicating whether merge processing of the predetermined block is to be applied and if so, identifying, for the respective combined merge syntax element, a certain block of two possible blocks, wherein the certain block has a predetermined relative locational relationship to the predetermined block, wherein if one or more of the combined merge syntax elements suggest merge processing of the predetermined block, the coding parameters of the certain block are to be adopted as the coding parameters for the predetermined block, for multiple color planes of the picture, and, if merge processing is not indicated by the combined merge syntax elements, inserting, for each of the multiple color planes of the picture, coding parameters for the predetermined block into the data stream.

17. A method for generating a data stream into which a picture is coded, comprising:

determining a first subset of syntax elements defining a spatial sub-division of an array of information samples representing a spatially sampling of the picture into a plurality of simply connected regions of different sizes by recursively multi-partitioning, and a second subset of syntax elements being disjoined from the first subset, defining a merging of predetermined simply connected regions with spatially neighboring simply connected regions, wherein the second subset of syntax elements includes combined merge syntax elements, each indicating whether merge processing of a predetermined simply connected region is to be applied and if so, identifying, for the respective combined merge syntax element, a certain spatially neighboring simply connected region of two spatially neighboring simply connected regions for merge processing, wherein the certain spatially neighboring simply connected region has a predetermined relative locational relationship to the predetermined simply connected region; and

coding the array of information samples into the data stream with inserting the first and second subsets of syntax elements into the data stream,

wherein coding parameters for the predetermined simply connected regions are indicated by way of the second subset of syntax elements as to be adopted or predicted from the certain neighboring simply connected regions with which the predetermined simply connected regions are merged for multiple color planes of the picture,

wherein a data stream inserter is configured to insert, for each of the multiple color planes, the coding parameters of simply connected regions which are not merged into the data stream.

18. The method according to claim 16 , wherein the data stream comprises depth information.

19. The method according to claim 16 , wherein the data stream has encoded thereinto a sequence of pictures with each picture comprising an array of luma samples along with two arrays of chroma samples, wherein a scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a scaling factor for a spatial resolution in the vertical direction.

20. The method according to claim 16 , wherein

the multiple color planes of the picture are encoded independently.

21. A method for decoding a data stream having encoded therein an array of information samples, wherein the method comprises:

receiving and decoding the data stream which has the array of information samples encoded thereinto in accordance with the method according to claim 16 .

22. A method for decoding a data stream having encoded therein a picture, wherein the method comprises:

receiving and decoding the data stream which has the picture encoded thereinto in accordance with the method according to claim 17 .

23. A method for decoding a data stream having encoded therein an array of information samples representing spatially sampled information of a color plane of a picture, wherein the method comprises:

receiving and decoding the data stream comprising payload data for each of a plurality of blocks into which the array of information samples is sub-divided, and coding parameters associated with the respective block so as to prescribe the way the picture is to be reconstructed from the payload data for the respective block, wherein

the data stream also comprises combined merge syntax elements for predetermined blocks, each of the combined merge syntax elements indicating whether merge processing of a predetermined block is to be applied and if so, identifying, for the respective combined merge syntax element, a certain block of two possible blocks, wherein the certain block has a predetermined relative locational relationship to the predetermined block, wherein the coding parameters of the certain block are to be adopted as the coding parameters for the predetermined block for multiple color planes of the picture, and

for each predetermined block for which the combined merge syntax element does not indicate merge processing, for each of the multiple color planes of the picture, coding parameters for the respective predetermined block.

24. A method for decoding a data stream having encoded therein a picture, wherein the method comprises:

receiving and decoding the data stream, the data stream comprising:

a first subset of syntax elements defining a spatial sub-division of an array of information samples representing a spatially sampling of the picture into a plurality of simply connected regions of different sizes by recursively multi-partitioning, and

a second subset of syntax elements being disjoined from the first subset, defining a merging of predetermined simply connected regions with spatially neighboring simply connected regions, wherein the second subset of syntax elements includes combined merge syntax element, each indicating whether merge processing of a predetermined simply connected region is to be applied and if so, identifying, for the respective combined merge syntax element, a certain spatially neighboring simply connected region of two spatially neighboring simply connected regions for merge processing, wherein the certain spatially neighboring simply connected region has a predetermined relative locational relationship to the predetermined simply connected region; and

wherein the array of information samples is coded into the data stream,

wherein coding parameters for the predetermined simply connected regions are indicated by way of the second subset of syntax elements as to be adopted or predicted from the certain neighboring simply connected regions with which the predetermined simply connected regions are merged for multiple color planes of the picture,

wherein the data stream comprises, for each of the multiple color planes, the coding parameters of simply connected regions which are not merged.

25. The method according to claim 21 , wherein the data stream comprises depth information.

26. The method according to claim 21 , wherein the data stream has encoded thereinto a sequence of pictures with each picture comprising an array of luma samples along with two arrays of chroma samples,

wherein a scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a scaling factor for a spatial resolution in the vertical direction.

27. The method according to claim 21 , wherein

the multiple color planes of the picture are decoded independently.

Assignments (4)
CHANGE OF NAME Recorded Jan 30, 2026
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 074536/0781 →
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0615 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2023
From: HELLE, PHILIPP; OUDIN, SIMON; WINKEN, MARTIN; MARPE, DETLEV; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 063298/0682 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2022
From: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
To: GE VIDEO COMPRESSION, LLC
Reel/Frame 061985/0257 →
Priority Claims (1)
EP 10159799 · Apr 13, 2010 · regional
Continuity (7)
Continuation 17346904 · Jun 14, 2021
Continuation 16907861 · Jun 22, 2020
Continuation 16578542 · Sep 23, 2019
Continuation 16261823 · Jan 30, 2019
Continuation 13650635 · Oct 12, 2012
Continuation PCTEP2011055795 · Apr 13, 2011
Related Publication 20230098693A1 · Mar 30, 2023
Cited By (2)
US 12,513,307 US 12,561,715