IP Library › Granted Patent US 12671836

Granted Patent B2

US 12671836 · App. 18/805,406 · Granted Jun 30, 2026

Encoding strategies for adaptive switching of color spaces, color sampling rates and/or bit depths

Inventors: Bin Li (Beijing, CN); Jizheng Xu (Beijing, CN); Feng Wu (Beijing, CN)

Assignee: Microsoft Technology Licensing, LLC

H04N19/59 H04N19/117 H04N19/132 H04N19/147 H04N19/176 H04N19/182 H04N19/186 H04N19/587 H04N19/51 H04N19/593

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12671836
App. No.: 18/805,406
Granted: Jun 30, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Innovations in adaptive encoding for units of a video sequence can improve coding efficiency. For example, some of the innovations relate to encoding that includes adaptive switching of color spaces between units within a video sequence. Other innovations relate encoding that includes adaptive switching of color sampling rates between units within a video sequence. Still other innovations relate encoding that includes adaptive switching of bit depths between units within a video sequence.

Claims (54)

1 . One or more non-transitory computer-readable media having stored thereon computer-executable instructions for causing one or more processing units, when programmed thereby, to perform encoding operations for a video encoder, the encoding operations comprising:

encoding video in a sequence, thereby producing encoded data, wherein the encoding the video includes:

determining that color space conversion operations between a first color space and a second color space are permitted, the second color space being different than the first color space;

setting a value of a syntax element, for a parameter set, that indicates the color space conversion operations between the first color space and the second color space are permitted; and

encoding multiple units of the video within the sequence, including, for a given unit among the multiple units:

performing rate-distortion analysis to evaluate encoding of the given unit in the first color space and encoding of the given unit in the second color space, including performing the color space conversion operations between the first color space and the second color space for the given unit;

selecting between the first color space and the second color space for the given unit depending on results of the rate-distortion analysis;

setting a flag value for the given unit, the flag value indicating the selection between the first color space and the second color space for the given unit; and

entropy coding the flag value using context-adaptive binary arithmetic coding; and

outputting the encoded data in a bitstream, the encoded data including the parameter set and the flag value for the given unit.

2 . The one or more computer-readable media of claim 1 , wherein the first color space is a YUV-type color space, and wherein the second color space is an RGB-type color space.

3 . The one or more computer-readable media of claim 2 , wherein the YUV-type color space is YCoCg color space, and wherein the color space conversion operations implement a matrix multiplication operation with addition, subtraction, and bit shifting operations to convert values for an R component, values for a G component, and values for a B component into values for a Y component, values for a Co component, and values for a Cg component.

4 . The one or more computer-readable media of claim 1 , wherein the multiple units are pictures of the sequence, slices of one of the pictures of the sequence, or blocks of one of the pictures of the sequence.

5 . The one or more computer-readable media of claim 1 , wherein the multiple units are transform blocks of a given picture of the sequence, and wherein the flag value is signaled as part of a transform unit syntax structure for the given unit.

6 . The one or more computer-readable media of claim 1 , wherein the multiple units are coding units of a given picture of the sequence, and wherein the flag value is signaled as part of a coding unit syntax structure for the given unit.

7 . The one or more computer-readable media of claim 1 , wherein the first color space is a default color space and the second color space is a non-default color space, and wherein the rate-distortion analysis is biased toward the default color space for the given unit.

8 . The one or more computer-readable media of claim 7 , wherein the rate-distortion analysis is biased towards the default color space for the given unit by scaling up a distortion cost for the non-default color space, scaling up a rate cost for the non-default color space, adding a distortion penalty for the non-default color space, adding a rate penalty for the non-default color space, using a larger Lagrangian multiplier factor for the non-default color space, scaling down a distortion cost for the default color space, scaling down a rate cost for the default color space, and/or using a smaller Lagrangian multiplier factor for the default color space.

9 . The one or more computer-readable media of claim 1 , wherein the performing the rate-distortion analysis includes:

determining a rate-distortion cost for the encoding of the given unit in the first color space using a rate cost for the encoding of the given unit in the first color space, a distortion cost for the encoding of the given unit in the first color space, and a first weighting factor; and

determining a rate-distortion cost for the encoding of the given unit in the second color space using a rate cost for the encoding of the given unit in the second color space, a distortion cost for the encoding of the given unit in the second color space, and a second weighting factor.

10 . The one or more computer-readable media of claim 9 , wherein the first weighting factor is different than the second weighting factor.

11 . The one or more computer-readable media of claim 1 , wherein the parameter set is a sequence parameter set for the sequence, the value of the syntax element indicating the color space conversion operations between the first color space and the second color space are permitted for the sequence, and wherein the flag value is signaled as part of a syntax structure for the given unit when there are residual values for the given unit.

12 . A computer system comprising:

a video encoder, implemented with one or more processing units of the computer system, configured to encode video in a sequence, thereby producing encoded data, by performing encoding operations that include:

determining that color space conversion operations between a first color space and a second color space are permitted, the second color space being different than the first color space;

setting a value of a syntax element, for a parameter set, that indicates the color space conversion operations between the first color space and the second color space are permitted; and

encoding multiple units of the video within the sequence, including, for a given unit among the multiple units:

evaluating one or more criteria for switching between encoding of the given unit in the first color space and encoding of the given unit in the second color space, including determining a type of video content for the given unit;

selecting between the first color space and the second color space for the given unit depending on results of the evaluating the one or more criteria;

setting a flag value for the given unit, the flag value indicating the selection between the first color space and the second color space for the given unit; and

entropy coding the flag value using context-adaptive binary arithmetic coding; and

a buffer, implemented with memory of the computer system, configured to store the encoded data for output in a bitstream, the encoded data including the parameter set and the flag value for the given unit.

13 . The computer system of claim 12 , wherein the first color space is a YUV-type color space, and wherein the second color space is an RGB-type color space.

14 . The computer system of claim 13 , wherein the first color space is YCoCg color space, and wherein the color space conversion operations implement a matrix multiplication operation with addition, subtraction, and bit shifting operations to convert values for an R component, values for a G component, and values for a B component into values for a Y component, values for a Co component, and values for a Cg component.

15 . The computer system of claim 12 , wherein the multiple units are transform blocks of a given picture of the sequence, and wherein the flag value is signaled as part of a transform unit syntax structure for the given unit.

16 . The computer system of claim 12 , wherein the multiple units are coding units of a given picture of the sequence, and wherein the flag value is signaled as part of a coding unit syntax structure for the given unit.

17 . The computer system of claim 12 , wherein the parameter set is a sequence parameter set for the sequence, the value of the syntax element indicating the color space conversion operations between the first color space and the second color space are permitted for the sequence, and wherein the flag value is signaled as part of a syntax structure for the given unit when there are residual values for the given unit.

18 . The computer system of claim 12 , wherein the selecting between the first color space and the second color space for the given unit includes:

determining whether the type of video content for the given unit is natural video or artificially created video;

if the type of video content for the given unit is natural video, selecting the first color space, wherein the first color space is a YUV-type color space; and

if the type of video content for the given unit is artificially created video, selecting the second color space, wherein the second color space is an RGB-type color space.

19 . The computer system of claim 12 , wherein the selecting between the first color space and the second color space for the given unit includes:

determining that the type of video content for the given unit is natural video and, responsive to the determining that the type of video content for the given unit is natural video, selecting the first color space, wherein the first color space is a YUV-type color space; or

determining that the type of video content for the given unit is artificially created video and, responsive to the determining that the type of video content for the given unit is artificially created video, selecting the second color space, wherein the second color space is an RGB-type color space.

20 . In a computer system that implements a video encoder, a method comprising: