Rate allocation for mixed content video
View Patent ↗The present invention relates to regulating the quality and/or bitrate of content within mixed content video when the video is compressed subject to a bitrate constraint. For example, a screen capture encoder encodes palletized content within a frame of screen capture video. Subject to an overall bitrate constraint, the encoder then allocates bits for continuous tone content within the frame. By controlling the allocation of bits used to encode the continuous tone content, the encoder regulates bitrate for the continuous tone content. This in turn can allow the encoder to regulate spatial quality and/or overall temporal quality for the video. In one scenario, for screen capture video encoded to a (relatively) constant overall bitrate, the screen capture encoder reduces the bitrate (and quality) of the continuous tone content, instead spending bits to increase the overall frame rate of the video.
1. In a screen capture encoder, a method for encoding captured screen content, the method comprising:
encoding one or more components of the captured screen content using lossless compression;
allocating bits for continuous tone pixels of the captured screen content, wherein the allocating comprises:
computing a preliminary bit allocation;
comparing the preliminary bit allocation with a number of available bits in view of the encoding using lossless compression; and
allocating the bits based at least in part on results of the comparing; and
encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.
2. The method of claim 1 wherein the one or more components of the captured screen content encoded using lossless compression include palettized pixels.
3. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include a segmentation map.
4. The method of claim 2 wherein the one or more components of the captured screen content encoded using lossless compression further include motion information and frame header information.
5. The method of claim 1 further comprising:
repeating the encoding the one or more components, the allocating, and the encoding the continuous tone pixels on a frame-by-frame basis for a sequence of plural frames.
6. In a screen capture encoder, a method for encoding captured screen content, the method comprising:
encoding one or more components of the captured screen content using lossless compression;
allocating bits for continuous tone pixels of the captured screen content, wherein the allocating comprises:
computing a preliminary bit allocation;
computing a minimum quality bit allocation;
comparing the preliminary bit allocation and the minimum quality bit allocation; and
allocating the bits based at least in part on results of the comparing; and
encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.
7. In a screen capture encoder, a method for encoding captured screen content, the method comprising:
encoding one or more components of the captured screen content using lossless compression;
allocating bits for continuous tone pixels of the captured screen content, wherein the allocating comprises:
estimating a number of bits needed to encode the captured screen content;
estimating a number of bits needed to encode the continuous tone pixels; and
computing the minimum of:
the difference between the estimated number of bits needed to encode the captured screen content and a number of bits spent in the encoding using lossless compression;
a number of available bits in view of the encoding using lossless compression; and
the estimated number of bits needed to encode the continuous tone pixels; and
encoding the continuous tone pixels, wherein the allocating regulates quality and bitrate for the continuous tone pixels.
8. The method of claim 7 wherein the number of available bits is further reduced according to a buffer filling factor.
9. The method of claim 8 wherein the buffer filling factor is fixed during encoding.
10. The method of claim 8 wherein the buffer filling factor is variable during encoding.
11. The method of claim 7 wherein the encoding the continuous tone pixels comprises using a combination of lossy compression and lossless compression.
12. A computer-readable medium storing computer-executable instructions for causing a computer programmed thereby to perform a method of claim 7 .
13. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform method of encoding a mixture of palettized screen capture content and continuous tone screen capture content, the method comprising:
encoding the palettized screen capture content;
encoding a frame header and segmentation map for a frame that includes the palettized screen capture content and the continuous tone screen capture content;
allocating bits for encoding the continuous tone screen capture content; and
encoding the continuous tone screen capture content, wherein the allocating controls bitrate for the continuous tone screen capture content.
14. The computer-readable medium of claim 13 wherein the encoding the palettized screen capture content uses lossless compression and wherein the encoding the continuous tone screen capture content uses lossy and lossless compression.
15. The computer-readable medium of claim 13 wherein the allocating depends at least in part upon preliminary and minimum quality bit allocations for the continuous tone screen capture content and a number of available bits.
16. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform method of encoding a mixture of palettized screen capture content and continuous tone screen capture content, the method comprising:
encoding the palettized screen capture content;
allocating bits for encoding the continuous tone screen capture content, wherein the allocating depends at least in part upon preliminary and minimum quality bit allocations for the continuous tone screen capture content and a number of available bits; and
encoding the continuous tone screen capture content, wherein the allocating controls bitrate for the continuous tone screen capture content.
17. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a frame comprising palettized content and continuous tone content, the method comprising:
encoding the palettized content in the frame;
checking for a number of bits available after the encoding the palettized content;
based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content, and wherein the allocating comprises:
computing a preliminary bit allocation;
comparing the preliminary bit allocation with the number of bits available after the encoding the palettized content; and
allocating the bits based at least in part on results of the comparing; and
encoding the continuous tone content in the frame with the second number of bits.
18. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method of encoding a frame comprising palettized content and continuous tone content, the method comprising:
encoding the palettized content in the frame;
checking for a number of bits available after the encoding the palettized content;
based at least in part on the checking, allocating a second number of bits, wherein the second number of bits is less than or equal to the number of bits available after the encoding the palettized content, and wherein the allocating comprises:
computing a preliminary bit allocation;
computing a minimum quality bit allocation;
comparing the preliminary bit allocation and the minimum quality bit allocation; and
allocating the bits based at least in part on results of the comparing; and
encoding the continuous tone content in the frame with the second number of bits.
19. The computer-readable medium of claim 18 wherein the method further comprises:
repeating the encoding the palettized content, the checking, the allocating, and the encoding the continuous tone content on a frame-by-frame basis for a sequence of plural frames.
20. The computer-readable medium of claim 18 wherein the encoding the palettized content uses lossless compression, and wherein the encoding the continuous tone content uses a combination of lossy compression and lossless compression.