IP Library › Granted Patent US 9,948,934
Granted Patent B2
US 9,948,934 · App. 14/322,720 · Granted Apr 17, 2018

Estimating rate costs in video encoding operations using entropy encoding statistics

Inventor: Jim C. Chou (San Jose, CA)
Assignee: Apple Inc.
H04N19/13H04N19/103H04N19/149H04N19/1887
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,948,934
App. No.
14/322,720
Granted
Apr 17, 2018
Kind
B2
Abstract

A component of an entropy encoding stage of a block processing pipeline (e.g., a CABAC encoder) may, for a block of pixels in a video frame, accumulate counts indicating the number of times each of two possible symbols is used in encoding a syntax element bin. An empirical probability for each symbol, an estimated entropy, and an estimated rate cost for encoding the bin may be computed, dependent on the symbol counts. A pipeline stage that precedes the entropy encoding stage may, upon receiving another block of pixels for the video frame, calculate and use the estimated rate cost when making encoding decisions for the other block of pixels based on a cost function that includes a rate cost term. The symbol counts or empirical probabilities may be passed to the earlier pipeline stage or written to a shared memory, from which components of the earlier stage may obtain them.

Claims (82)

1. An apparatus, comprising:

a block processing pipeline configured to process blocks of pixels from a video frame; and

a memory accessible by respective processors in components at each of two or more stages of the block processing pipeline;

wherein an entropy encoding stage of the block processing pipeline comprises a binary arithmetic coding component configured to encode syntax element bins that represent the blocks of pixels;

wherein, for each of one or more of a plurality of syntax element bins for a given block of pixels, the binary arithmetic coding component is configured to:

code the syntax element bin using one or more symbols, each of which has one of two possible values;

update, in the memory, at least one of: a count of symbols used in coding the syntax element bin having one of the two possible values or a count of symbols used in coding the syntax element bin having the other one of the two possible values; and

wherein, when processing another block of pixels, a processor in a component at a stage of the block processing pipeline that precedes the entropy encoding stage is configured to:

access the count of symbols used in coding a given syntax element bin having the one of the two possible values and the count of symbols used in coding the given syntax element bin having the other one of the two possible values;

compute, for the given syntax element bin, a current probability for each of the two possible values of the symbols used to encode the given syntax element bin, dependent on the accessed counts;

compute an estimate of entropy for the given syntax element bin, dependent on the computed current probability for each of the two possible values; and

calculate an estimated rate cost for encoding the given syntax element bin for the other block of pixels, wherein the estimated rate cost is dependent on the computed estimate of entropy, and wherein the estimated rate cost represents an estimate of the amount of data needed to encode the given syntax element bin of the other block of pixels.

2. The apparatus of claim 1 , wherein the binary arithmetic coding component is configured to perform context-adaptive binary arithmetic coding.

3. The apparatus of claim 1 ,

wherein the stage of the block processing pipeline that precedes the entropy encoding stage is a motion estimation stage; and

wherein the given syntax element bin is one of a plurality of syntax element bins that represent a difference between a motion vector candidate that is being evaluated and a motion vector predictor.

4. The apparatus of claim 3 , wherein, when processing the other block of pixels, the processor in the component at the motion estimation stage is further configured to:

calculate an estimated cost associated with encoding the given syntax element bin when the motion vector candidate being evaluated is selected, dependent on the estimated rate cost; and

select one of a plurality of candidate motion vectors that comprises the motion vector candidate being evaluated, dependent on the estimated cost associated with encoding the given syntax element bin when the motion vector candidate being evaluated is selected.

5. The apparatus of claim 1 ,

wherein, when processing the other block of pixels, the processor in the component at the stage of the block processing pipeline that precedes the entropy encoding stage is further configured to calculate an estimated cost associated with encoding the given syntax element bin under particular conditions, dependent on the estimated rate cost; and

wherein to calculate the estimated cost associated with encoding the given syntax element bin under the particular conditions, the processor is configured to apply a cost function that is dependent on the estimated rate cost and a distortion term.

6. The apparatus of claim 1 ,

wherein the stage of the block processing pipeline that precedes the entropy encoding stage is an intra estimation stage; and

wherein, when processing the other block of pixels, the processor in the component at the intra estimation stage is further configured to:

calculate an estimated cost associated with encoding the given syntax element bin in a particular intra estimation mode, dependent on the estimated rate cost; and

select one of a plurality of candidate intra estimation modes that comprises the particular intra estimation mode, dependent on the estimated cost associated with encoding the given syntax element bin in the particular intra estimation mode.

7. The apparatus of claim 1 ,

wherein the stage of the block processing pipeline that precedes the entropy encoding stage is a mode decision stage; and

wherein, when processing the other block of pixels, the processor in the component at the mode decision stage is further configured to:

calculate an estimated cost associated with encoding the given syntax element bin in a particular encoding mode, dependent on the estimated rate cost; and

select one of a plurality of candidate encoding modes that comprises the particular encoding mode, dependent on the estimated cost associated with encoding the given syntax element bin in the particular encoding mode.

8. The apparatus of claim 1 ,

wherein the estimated rate cost is further dependent on an inefficiency parameter; and

wherein to calculate the estimated rate cost, the processor in the component at the stage of the block processing pipeline that precedes the entropy encoding stage is configured to multiply the computed estimate of entropy by the inefficiency parameter.

9. A method, comprising:

inputting data representing a block of pixels from a video frame to a video encoding pipeline comprising a plurality of stages, wherein each stage of the pipeline is configured to perform at least one operation on blocks of pixels passing through the pipeline, and wherein the input data comprises a plurality of syntax element bins for the block of pixels; and

performing, by one or more stages of the pipeline:

coding a given one of the plurality of syntax element bins using one or more symbols, each of which has one of two possible values;

incrementing at least one count value associated with the encoding of the given syntax element bin to reflect the values of the one or more symbols used in encoding the given syntax element bin;

receiving input data representing a second block of pixels from the video frame, wherein the input data comprises a plurality of syntax element bins for the second block of pixels, wherein the plurality of syntax element bins for the second block of pixels comprises the given syntax element bin; and

prior to coding the given syntax element bin for the second block of pixels:

accessing the at least one count value; and

calculating an estimated rate cost for encoding the given syntax element bin, wherein the estimated rate cost is dependent on the at least one count value, and wherein the estimated rate cost represents an estimate of the amount of data needed to encode the given syntax element bin for the second block of pixels.

10. The method of claim 9 , wherein said calculating comprises:

computing, for the given syntax element bin, a current probability for each of the two possible values of the symbols used to encode the given syntax element bin, dependent on the at least one count value; and

computing an estimate of entropy for the given syntax element bin, dependent on the computed current probability for each of the two possible values.

11. The method of claim 9 , further comprising:

calculating an estimated cost of encoding the given syntax element bin for the second block of pixels in a particular encoding mode, dependent on the estimated rate cost; and

selecting one of a plurality of candidate encoding modes in which to encode the given syntax element bin for the second block of pixels, wherein the plurality of candidate encoding modes comprises the particular encoding mode, and wherein said selecting is dependent on the estimated cost of encoding the given syntax element bin for the second block of pixels in the particular encoding mode.

12. The method of claim 11 , wherein said selecting one of a plurality of candidate encoding modes comprises selecting one of a plurality of intra modes during intra estimation.

13. The method of claim 11 , wherein said calculating an estimated cost comprises applying a cost function that is dependent on the estimated rate cost and a distortion term.

14. The method of claim 9 ,

wherein the given syntax element bin is one of a plurality of syntax element bins that represent a difference between a motion vector candidate that is being evaluated and a motion vector predictor; and

wherein the method further comprises:

calculating an estimated cost associated with encoding the given syntax element bin when the motion vector candidate being evaluated is selected, dependent on the estimated rate cost; and

selecting one of a plurality of candidate motion vectors that includes the motion vector candidate being evaluated, dependent on the estimated cost associated with encoding the given syntax element bin when the motion vector candidate being evaluated is selected.

15. The method of claim 9 , further comprising:

coding the given syntax element bin for the second block of pixels using one or more symbols, each of which has one of two possible values;

updating the at least one count value to reflect the values of the one or more symbols used in encoding the given syntax element bin for the second block of pixels;

receiving input data representing a third block of pixels from the video frame, wherein the input data comprises a plurality of syntax element bins for the third block of pixels, wherein the plurality of syntax element bins for the third block of pixels comprises the given syntax element bin; and

prior to coding the given syntax element bin for the third block of pixels, and subsequent to said updating:

accessing the at least one count value; and

calculating an updated estimated rate cost for encoding the given syntax element bin, wherein the updated estimated rate cost is dependent on the at least one count value, and wherein the updated estimated rate cost represents an estimate of the amount of data needed to encode the given syntax element bin for the third block of pixels.

16. The method of claim 15 , further comprising:

subsequent to coding the given syntax element for a last one of a pre-determined number of blocks of pixels in the video frame, resetting the at least one count value to an initial value.

17. The method of claim 9 , wherein the given syntax element bin represents a quantized coefficient for the block of pixels that was generated in a stage of a block processing pipeline that precedes an entropy encoding stage that performs said coding.

18. A device, comprising:

a memory; and

an apparatus configured to process video frames and to store the processed video frames as frame data to the memory;

wherein the apparatus is configured to:

receive input data representing a given block of pixels from a video frame, wherein the input data comprises a plurality of syntax element bins for the given block of pixels;

access a data structure in the memory, wherein the data structure stores, for a given one of the plurality of syntax element bins for the given block of pixels, a current probability for each of two possible symbol values to be used in encoding the given syntax element bin;

compute an estimate of entropy for the given syntax element bin, dependent on the current probability for each of the two possible symbol values;

calculate an estimated rate cost for encoding the given syntax element bin for the given block of pixels, wherein the estimated rate cost is dependent on the computed estimate of entropy, and wherein the estimated rate cost represents an estimate of the amount of data needed to encode the given syntax element bin for the given block of pixels.

19. The device of claim 18 , wherein the apparatus is further configured to:

calculate an estimated cost of encoding the given syntax element bin for the given block of pixels in a particular encoding mode, dependent on the estimated rate cost; and

select one of a plurality of candidate encoding modes in which to encode the given syntax element bin for the given block of pixels, wherein the plurality of candidate encoding modes comprises the particular encoding mode, and wherein selection of the one of the plurality of candidate encoding modes is dependent on the estimated cost of encoding the given syntax element bin for the given block of pixels in the particular encoding mode.

20. The device of claim 18 , wherein, prior to receiving the input data representing the given block of pixels, the apparatus is configured to:

accumulate a respective count value indicating a number of times that each of two possible symbols was used in encoding the given syntax element bin for one or more other block of pixels from the video frame when processing previously received input data representing the one or more other blocks of pixels from the video frame;

compute the current probability for each of the two possible symbol values to be used in encoding the given syntax element bin, dependent on the respective count values; and

store the current probability for each of the two possible symbol values to be used in encoding the given syntax element bin in the data structure in the memory.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2014
From: CHOU, JIM C.
To: APPLE INC.
Reel/Frame 033239/0702 →
Continuity (1)
Related Publication 20160007046A1 · Jan 7, 2016