IP Library › Granted Patent US 11,317,101
Granted Patent B2
US 11,317,101 · App. 14/921,378 · Granted Apr 26, 2022

Inter frame candidate selection for a video encoder

Inventor: Juha Pekka Maaninen (Oulu, FI)
H04N19/176H04N19/105H04N19/107H04N19/109H04N19/154H04N19/436H04N19/513
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,317,101
App. No.
14/921,378
Filed
Oct 23, 2015
Granted
Apr 26, 2022
Kind
B2
Examiner
LEE, JIMMY S
Art Unit
2483
USPC
375/240.16
Abstract

Inter frame candidate selection may include identifying a current block from a current input frame from an input video stream, and generating an encoded block by encoding the current block, wherein encoding the current block includes determining an inter-coding candidate motion vector. Determining the inter-coding candidate motion vector may include identifying a plurality of motion vectors, wherein the plurality of motion vectors includes a context motion vector identified from a block neighboring the current block in the current input frame, a zero valued motion vector, and an estimated motion vector based on the current block and a reference frame, determining a plurality of cost values by determining a cost value for each respective motion vector from the plurality of motion vectors, and identifying a motion vector from the plurality of motion vectors having a minimal cost value as the inter-coding candidate motion vector.

Claims (105)

1. A method, comprising:

identifying a current block from a current input frame from an input video stream;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded block by encoding the current block, wherein encoding the current block includes determining an inter-coding candidate motion vector, wherein determining the inter-coding candidate motion vector includes:

identifying a plurality of motion vectors, wherein the plurality of motion vectors includes a context motion vector identified from a block neighboring the current block in the current input frame, a zero valued motion vector, and an estimated motion vector based on the current block and a reference frame,

determining a plurality of cost values by determining a cost value for each respective motion vector from the plurality of motion vectors, wherein determining the cost value for a motion vector from the plurality of motion vectors includes:

determining a distortion measurement for encoding the current block using the motion vector;

determining an estimated encoding cost for encoding the current block using the motion vector;

identifying a weighting value; and

determining the cost value as a sum of the distortion measurement and a product of the weighting value and the estimated encoding cost, and

identifying a motion vector from the plurality of motion vectors having a minimal cost value as the inter-coding candidate motion vector;

including the encoded block in an output bitstream; and

storing or transmitting the output bitstream.

2. The method of claim 1 , wherein identifying the plurality of motion vectors includes generating the estimated motion vector, wherein generating the estimated motion vector includes:

generating a first candidate estimated motion vector based on the current block;

generating a second candidate estimated motion vector based on a first sub-block of the current block;

generating a third candidate estimated motion vector based on a second sub-block of the current block;

determining a first motion estimation cost value as a sum of a first penalty value and sum of absolute differences for the first candidate estimated motion vector;

determining a second motion estimation cost value as a sum of a second penalty value and sum of absolute differences for the second candidate estimated motion vector and the third candidate estimated motion vector;

selecting the first candidate estimated motion vector as the estimated motion vector on a condition that the second motion estimation cost value exceeds the first motion estimation cost value; and

selecting the second candidate estimated motion vector as the estimated motion vector on a condition that the first motion estimation cost value exceeds the second motion estimation cost value.

3. The method of claim 1 , wherein identifying the motion vector from the plurality of motion vectors having the minimal cost value as the inter-coding candidate motion vector includes:

selecting the context motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the zero valued motion vector exceeds the cost value associated with the context motion vector and the cost value associated with the estimated motion vector exceeds the cost value associated with the context motion vector;

selecting the zero valued motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the context motion vector exceeds the cost value associated with the zero valued motion vector and the cost value associated with the estimated motion vector exceeds the cost value associated with the zero valued motion vector;

selecting the estimated motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the zero valued motion vector exceeds the cost value associated with the estimated motion vector and the cost value associated with the context vector exceeds the cost value associated with the estimated motion vector.

4. The method of claim 1 , wherein determining the cost value for the motion vector from the plurality of motion vectors includes:

determining a predicted block based on the current block and the motion vector;

determining a residual block as a difference between the current block and the predicted block;

generating a transform block based on the residual block; and

generating a quantized block based on the transform block.

5. The method of claim 4 , wherein determining the distortion measurement for encoding the current block using the motion vector includes:

determining an inverse-quantized block based on the quantized block;

determining a sum of squared errors between the transform block and the inverse-quantized block as the distortion measurement.

6. The method of claim 4 , wherein determining the estimated encoding cost for encoding the current block using the motion vector includes:

identifying a defined scan order for the quantized block, wherein the quantized block includes a plurality of quantized transform coefficients;

determining a plurality of estimated coefficient encoding costs by determining an estimated coefficient encoding cost for each respective quantized transform coefficient from the plurality of quantized transform coefficients; and

determining a sum of the plurality of estimated coefficient encoding costs as the estimated encoding cost.

7. The method of claim 1 , wherein encoding the current block includes determining an intra-coding candidate motion vector.

8. The method of claim 7 , wherein encoding the current block includes:

determining a first sum of absolute differences for the intra-coding candidate motion vector;

determining a second sum of absolute differences for the inter-coding candidate motion vector;

encoding the current block using the intra-coding candidate motion vector on a condition that the second sum of absolute differences exceeds the first sum of absolute differences; and

encoding the current block using the inter-coding candidate motion vector on a condition that the first sum of absolute differences exceeds the second sum of absolute differences.

9. The method of claim 1 , wherein determining plurality of cost values includes determining each cost value from the plurality of cost values in parallel.

10. The method of claim 1 , wherein identifying the context motion vector includes identifying the context motion vector such that the context motion vector is a non-zero motion vector.

11. The method of claim 1 , wherein identifying the context motion vector includes identifying the context motion vector from the block neighboring the current block in the current input frame on a condition that the block neighboring the current block in the current input frame is a previously coded block encoded using the context motion vector.

12. The method of claim 1 , wherein determining the inter-coding candidate motion vector includes:

identifying a second context motion vector from a second block neighboring the current block in the current input frame.

13. The method of claim 12 , wherein identifying the second context motion vector includes identifying the second context motion vector such that the second context motion vector is a non-zero motion vector.

14. The method of claim 13 , wherein identifying the second context motion vector includes identifying the second context motion vector from the second block neighboring the current block in the current input frame on a condition that the second block neighboring the current block in the current input frame is a previously coded block encoded using the second context motion vector.

15. A method, comprising:

identifying a current block from a current input frame from an input video stream;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded block by encoding the current block, wherein encoding the current block includes determining an inter-coding candidate motion vector, wherein determining the inter-coding candidate motion vector includes:

identifying a plurality of motion vectors, wherein the plurality of motion vectors includes a context motion vector identified from a block neighboring the current block in the current input frame, a zero valued motion vector, and an estimated motion vector based on the current block and a reference frame,

determining a plurality of cost values by determining a cost value for each respective motion vector from the plurality of motion vectors, wherein determining the cost value for a motion vector from the plurality of motion vectors includes:

determining a predicted block based on the current block and the motion vector;

determining a residual block as a difference between the current block and the predicted block;

generating a transform block based on the residual block;

generating a quantized block based on the transform block;

determining an inverse-quantized block based on the quantized block;

determining a sum of squared errors between the transform block and the inverse-quantized block as a distortion measurement for encoding the current block using the motion vector;

identifying a defined scan order for the quantized block, wherein the quantized block includes a plurality of quantized transform coefficients;

determining a plurality of estimated coefficient encoding costs by determining an estimated coefficient encoding cost for each respective quantized transform coefficient from the plurality of quantized transform coefficients;

determining a sum of the plurality of estimated coefficient encoding costs as an estimated encoding cost for encoding the current block using the motion vector;

identifying a weighting value; and

determining the cost value as a sum of the distortion measurement and a product of the weighting value and the estimated encoding cost, and

identifying a motion vector from the plurality of motion vectors having a minimal cost value as the inter-coding candidate motion vector;

including the encoded block in an output bitstream; and

storing or transmitting the output bitstream.

16. The method of claim 15 , wherein identifying the plurality of motion vectors includes generating the estimated motion vector, wherein generating the estimated motion vector includes:

generating a first candidate estimated motion vector based on the current block;

generating a second candidate estimated motion vector based on a first sub-block of the current block;

generating a third candidate estimated motion vector based on a second sub-block of the current block;

determining a first motion estimation cost value as a sum of a first penalty value and sum of absolute differences for the first candidate estimated motion vector;

determining a second motion estimation cost value as a sum of a second penalty value and sum of absolute differences for the second candidate estimated motion vector and the third candidate estimated motion vector;

selecting the first candidate estimated motion vector as the estimated motion vector on a condition that the second motion estimation cost value exceeds the first motion estimation cost value; and

selecting the second candidate estimated motion vector as the estimated motion vector on a condition that the first motion estimation cost value exceeds the second motion estimation cost value.

17. The method of claim 15 , wherein identifying the motion vector from the plurality of motion vectors having the minimal cost value as the inter-coding candidate motion vector includes:

selecting the context motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the zero valued motion vector exceeds the cost value associated with the context motion vector and the cost value associated with the estimated motion vector exceeds the cost value associated with the context motion vector;

selecting the zero valued motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the context motion vector exceeds the cost value associated with the zero valued motion vector and the cost value associated with the estimated motion vector exceeds the cost value associated with the zero valued motion vector;

selecting the estimated motion vector as the inter-coding candidate motion vector on a condition that the cost value associated with the zero valued motion vector exceeds the cost value associated with the estimated motion vector and the cost value associated with the context vector exceeds the cost value associated with the estimated motion vector.

18. The method of claim 15 , wherein encoding the current block includes:

determining an intra-coding candidate motion vector;

determining a first sum of absolute differences for the intra-coding candidate motion vector;

determining a second sum of absolute differences for the inter-coding candidate motion vector;

encoding the current block using the intra-coding candidate motion vector on a condition that the second sum of absolute differences exceeds the first sum of absolute differences; and

encoding the current block using the inter-coding candidate motion vector on a condition that the first sum of absolute differences exceeds the second sum of absolute differences.

19. The method of claim 15 , wherein determining plurality of cost values includes determining each cost value from the plurality of cost values in parallel.

20. A method, comprising:

identifying a current block from a current input frame from an input video stream;

generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded block by encoding the current block, wherein encoding the current block includes:

determining an inter-coding candidate motion vector, wherein determining the inter-coding candidate motion vector includes:

identifying a plurality of motion vectors, wherein the plurality of motion vectors includes a context motion vector identified from a block neighboring the current block in the current input frame, a zero valued motion vector, and an estimated motion vector based on the current block and a reference frame;

determining a plurality of cost values by determining a cost value for each respective motion vector from the plurality of motion vectors, wherein determining the cost value for a motion vector from the plurality of motion vectors includes:

determining a distortion measurement for encoding the current block using the motion vector,

determining an estimated encoding cost for encoding the current block using the motion vector,

identifying a weighting value, and

determining the cost value as a sum of the distortion measurement and a product of the weighting value and the estimated encoding cost; and

identifying a motion vector from the plurality of motion vectors having a minimal cost value as the inter-coding candidate motion vector,

determining an intra-coding candidate motion vector,

determining a first sum of absolute differences for the intra-coding candidate motion vector,

determining a second sum of absolute differences for the inter-coding candidate motion vector,

encoding the current block using the intra-coding candidate motion vector on a condition that the second sum of absolute differences exceeds the first sum of absolute differences, and

encoding the current block using the inter-coding candidate motion vector on a condition that the first sum of absolute differences exceeds the second sum of absolute differences;

including the encoded block in an output bitstream; and

storing or transmitting the output bitstream.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2015
From: MAANINEN, JUHA PEKKA
To: GOOGLE INC.
Reel/Frame 037056/0515 →
Continuity (2)
Continuation In Part 13494961 · Jun 12, 2012
Related Publication 20160044332A1 · Feb 11, 2016