IP Library › Granted Patent US 10,321,162
Granted Patent B2
US 10,321,162 · App. 15/175,944 · Granted Jun 11, 2019

Derivation of color gamut scalability parameters and tables in scalable video coding

Inventors: Adarsh Krishnan Ramasubramonian (San Diego, CA); Xiang Li (San Diego, CA); Joel Sole Rojals (San Diego, CA); Marta Karczewicz (San Diego, CA); Sungwon Lee (San Diego, CA); Dmytro Rusanovskyy (San Diego, CA); Done Bugdayci Sansli (Tampere, FI)
Assignee: Qualcomm Incorporated
H04N19/86H04N19/103H04N19/105H04N19/119H04N19/13H04N19/147H04N19/176H04N19/186H04N19/187H04N19/30H04N19/36H04N19/50H04N19/587H04N19/615H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,321,162
App. No.
15/175,944
Granted
Jun 11, 2019
Kind
B2
Abstract

Techniques are described for identifying and reducing the incidence of artifacts in video using color gamut scalability (CGS) parameters and tables in scalable video coding (SVC). Derivation of CGS mapping tables are performed for each partition of pixel values in a color space. The pixel value domain is split into partitions and each is optimized independently. Color prediction techniques for CGS may be used by video encoders and/or video decoders to generate inter-layer reference pictures when a color gamut for a lower layer of video data is different than a color gamut for a higher layer of the video data. When mapped values are used as inter-layer predication references for the enhancement layer blocks, artifacts may appear in some frames of the sequences. A video encoder may identify blocks that potentially contain these artifacts and disable inter-layer prediction in those identified blocks.

Claims (89)

1. A method of encoding video data, the method comprising:

receiving video data comprising a plurality of video blocks, the plurality of video blocks comprising a plurality of pixel values in a color space;

obtaining color gamut scalability (CGS) parameters comprising splitting the color space into a plurality of rectangular cuboid partitions, each of the plurality of rectangular cuboid partitions comprising at least one partition boundary between the plurality of rectangular cuboid partitions, and wherein the at least one partition boundary comprises a partition boundary of a pixel component in the color space;

defining a first threshold value of pixel values for a first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

defining a second threshold value of pixel values for a second rectangular cuboid partition of the plurality of rectangular cuboid partitions different from the first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

identifying pixel values in the plurality of rectangular cuboid partitions that comprise characteristics that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks;

identifying blocks of the plurality of video blocks that contain the pixel values that comprise characteristics that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, wherein identifying blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks comprises:

collecting all pixel values between the first threshold value of pixel values for the first rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary, and

collecting all pixel values between the second threshold value of pixel values for the second rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary;

disabling an inter-layer prediction mode for the identified blocks; and

encoding the plurality of video blocks using a scalable video coding process with the obtained CGS parameters.

2. The method of claim 1 , wherein the plurality of video blocks comprises a coding unit or a prediction unit.

3. The method of claim 1 , wherein the pixel component comprises a luma component of the color space.

4. The method of claim 1 , wherein identifying pixel values in the rectangular cuboid partitions that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks comprises collecting all pixel values that fall within a threshold value of pixel values for a first partition boundary into a first list of pixel values and a second partition boundary into a second list of pixel values.

5. The method of claim 1 , wherein the CGS parameters comprises a CGS map corresponding to the rectangular cuboid partition of the plurality of rectangular cuboid partitions that comprises a maximum number of pixels in a first video block of the plurality of video blocks used to map all pixels in the first video block.

6. The method of claim 1 wherein obtaining CGS parameters further comprises:

estimating CGS parameters of a rectangular cuboid partition of the plurality of rectangular cuboid partitions based on all pixel values in the rectangular cuboid partition;

calculating a plurality of difference values between each mapped value of the CGS parameters and an original enhancement layer reference value;

excluding at least one pixel value of the all pixel values in the rectangular cuboid partition where the difference value of the plurality of difference values exceeds a threshold; and

re-estimating the CGS parameters of the rectangular cuboid partition based on non-excluded pixel values in the rectangular cuboid partition.

7. The method of claim 1 , wherein splitting the color space into a plurality of rectangular cuboid partitions comprises:

calculating a CGS bit budget based, at least in part, on a ratio of a number of bits corresponding to CGS parameters that have been sent until a current picture to a total number of bits sent until the current picture;

when the CGS bit budget is below a first threshold, increasing a number of the plurality of rectangular cuboid partitions by a first value compared with a second number of rectangular cuboid partitions of a previous picture, the first value determined based on the number of plurality of rectangular cuboid partitions;

when the CGS bit budget is above a second threshold, decreasing a number of the plurality of rectangular cuboid partitions by a second value compared with a second number of rectangular cuboid partitions of a previous picture, the second value determined based on the number of plurality of rectangular cuboid partitions; and

when the CGS bit budget is above or equal to the first threshold or below or equal to the second threshold, changing the number of the plurality of rectangular cuboid partitions by a third value compared with a second number of rectangular cuboid partitions of a previous picture, the third value determined based on the number of plurality of rectangular cuboid partitions.

8. The method of claim 1 , wherein the color space comprises a YUV or YC B C R color space.

9. The method of claim 1 , further comprising:

defining a third threshold value of pixel values for the at least one partition boundary,

wherein identifying blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks comprises collecting all pixel values that fall within the third threshold value of pixel values for a rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary into a list of pixel values.

10. The method of claim 1 , wherein:

obtaining color gamut scalability (CGS) parameters further comprises:

generating a three dimensional lookup table based on the plurality of rectangular cuboid partitions for each color component, and

minimizing a mean squared value between pixels belonging to each of the plurality of rectangular cuboid partitions, and

encoding the plurality of video blocks further comprises:

for each of the plurality of rectangular cuboid partitions, encoding CGS parameters for linear color mapping function of color values in the three dimensional lookup table.

11. A device for processing data, the device comprising:

a memory configured to store video data; and

one or more processors in communication with the memory and configured to:

receive the video data comprising a plurality of video blocks, the plurality of video blocks comprising a plurality of pixel values in a color space;

obtain color gamut scalability (CGS) parameters via a split of the color space into a plurality of rectangular cuboid partitions, each of the plurality of rectangular cuboid partitions comprising at least one partition boundary between the plurality of rectangular cuboid partitions, and wherein the at least one partition boundary comprises a partition boundary of a pixel component in the color space;

define a first threshold value of pixel values for a first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

define a second threshold value of pixel values for a second rectangular cuboid partition of the plurality of rectangular cuboid partitions different from the first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

identify pixel values in the rectangular cuboid partitions that comprise characteristics that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks;

identify blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, wherein to identify blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, the one or more processors are further configured to:

collect all pixel values between the first threshold value of pixel values for the first rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary, and

collect all pixel values between the second threshold value of pixel values for the second rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary;

disable an inter-layer prediction mode for the identified blocks; and

encode the plurality of video blocks using a scalable video coding process with the obtained CGS parameters.

12. The device of claim 11 , wherein the pixel component comprises a luma component of the color space.

13. The device of claim 11 , wherein the CGS parameters comprises a CGS map corresponding to the rectangular cuboid partition of the plurality of rectangular cuboid partitions that comprises a maximum number of pixels in a first video block of the plurality of video blocks used to map all pixels in the first video block.

14. The device of claim 11 , wherein the one or more processors are further configured to:

estimate CGS parameters of a rectangular cuboid partition of the plurality of rectangular cuboid partitions based on all pixel values in the rectangular cuboid partition;

calculate a plurality of difference values between each mapped value of the CGS parameters and an original enhancement layer reference value;

exclude at least one pixel value of the all pixel values in the rectangular cuboid partition where the difference value of the plurality of difference values exceeds a threshold; and

re-estimate the CGS parameters of the rectangular cuboid partition based on non-excluded pixel values in the rectangular cuboid partition.

15. The device of claim 11 , wherein the one or more processors are further configured to:

calculate a CGS bit budget based, at least in part, on a ratio of a number of bits corresponding to CGS parameters that have been sent until a current picture to a total number of bits sent until the current picture;

when the CGS bit budget is below a first threshold, increase a number of the plurality of rectangular cuboid partitions by a first value compared with a second number of rectangular cuboid partitions of a previous picture, the first value determined based on the number of plurality of rectangular cuboid partitions;

when the CGS bit budget is above a second threshold, decrease a number of the plurality of rectangular cuboid partitions by a second value compared with a second number of rectangular cuboid partitions of a previous picture, the second value determined based on the number of plurality of rectangular cuboid partitions; and

when the CGS bit budget is above or equal to the first threshold or below or equal to the second threshold, change the number of the plurality of rectangular cuboid partitions by a third value compared with a second number of rectangular cuboid partitions of a previous picture, the third value determined based on the number of plurality of rectangular cuboid partitions.

16. The device of claim 11 , wherein the one or more processor are further configured to:

define a third threshold value of pixel values for the at least one partition boundary; and

collect all pixel values that fall within the third threshold value of pixel values for a rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary into a list of pixel values.

17. The device of claim 11 , wherein the one or more processors are further configured to:

generate a three dimensional lookup table based on the plurality of rectangular cuboid partitions for each color component;

minimize of a mean squared value between pixels belonging to each of the plurality of rectangular cuboid partitions; and

encode CGS parameters for linear color mapping function of color values in the three dimensional lookup table.

18. A device for encoding video data, the device comprising:

means for receiving video data comprising a plurality of video blocks, the plurality of video blocks comprising a plurality of pixel values in a color space;

means for obtaining color gamut scalability (CGS) parameters comprising splitting the color space into a plurality of rectangular cuboid partitions, each of the plurality of rectangular cuboid partitions comprising at least one partition boundary between the plurality of rectangular cuboid partitions, and wherein the at least one partition boundary comprises a partition boundary of a pixel component in the color space;

means for defining a first threshold value of pixel values for a first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

means for defining a second threshold value of pixel values for a second rectangular cuboid partition of the plurality of rectangular cuboid partitions different from the first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

means for identifying pixel values in the rectangular cuboid partitions that comprise characteristics that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks;

means for identifying blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, wherein the means for identifying blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks comprises:

means for collecting all pixel values between the first threshold value of pixel values for the first rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary, and

means for collecting all pixel values between the second threshold value of pixel values for the second rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary;

means for disabling an inter-layer prediction mode for the identified blocks; and

means for encoding the plurality of video blocks using a scalable video coding process with the obtained CGS parameters.

19. A non-transitory computer-readable storage medium including instructions stored thereon that, when executed, cause at least one processor of to:

receive video data comprising a plurality of video blocks, the plurality of video blocks comprising a plurality of pixel values in a color space;

obtain color gamut scalability (CGS) parameters via a split of the color space into a plurality of rectangular cuboid partitions, each of the plurality of rectangular cuboid partitions comprising at least one partition boundary between the plurality of rectangular cuboid partitions, and wherein the at least one partition boundary comprises a partition boundary of a pixel component in the color space;

define a first threshold value of pixel values for a first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

define a second threshold value of pixel values for a second rectangular cuboid partition of the plurality of rectangular cuboid partitions different from the first rectangular cuboid partition of the plurality of rectangular cuboid partitions adjacent to the at least one partition boundary;

identify pixel values in the rectangular cuboid partitions that comprise characteristics that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks;

identify blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, wherein to identify blocks of the plurality of video blocks that contain the pixel values that may cause the obtained CGS parameters to produce artifacts when decoding the plurality of video blocks, the instructions further cause the at least one processor to:

collect all pixel values between the first threshold value of pixel values for the first rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary, and

collect all pixel values between the second threshold value of pixel values for the second rectangular cuboid partition of the plurality of rectangular cuboid partitions and the at least one partition boundary;

disable an inter-layer prediction mode for the identified blocks; and

encode the plurality of video blocks using a scalable video coding process with the obtained CGS parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2016
From: RAMASUBRAMONIAN, ADARSH KRISHNAN; LI, XIANG; SOLE ROJALS, JOEL; KARCZEWICZ, MARTA; LEE, SUNGWON; RUSANOVSKYY, DMYTRO; BUGDAYCI SANSLI, DONE
To: QUALCOMM INCORPORATED
Reel/Frame 038976/0624 →
Continuity (2)
Provisional Application 62172779 · Jun 8, 2015
Related Publication 20160360235A1 · Dec 8, 2016
Cited By (1)
US 12,382,063