IP Library › Granted Patent US 9,788,010
Granted Patent B2
US 9,788,010 · App. 12/773,145 · Granted Oct 10, 2017

Reducing computational complexity when video encoding uses bi-predictively encoded frames

Inventors: Soyeb Nagori (Bangalore, IN); Arun Shankar Kudana (Bangalore, IN); Pramod Kumar Swami (Bangalore, IN)
Assignee: TEXAS INSTRUMENTS INCORPORATED
H04N19/523H04N19/105H04N19/109H04N19/114H04N19/117H04N19/147H04N19/156H04N19/157H04N19/172H04N19/176H04N19/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,788,010
App. No.
12/773,145
Granted
Oct 10, 2017
Kind
B2
Abstract

Several techniques aimed at reducing computational complexity when encoding uses bi-predictively encoded frames (B-frames) are implemented in a video encoder. In an embodiment, B-frames are not used as reference frames for encoding P-frames and other B-frames. Non-use of B-frames allows a de-blocking filter used in the video encoder to be switched off when reconstructing encoded B-frames, and use of a lower complexity filter for fractional-resolution motion search for B-frames. In another embodiment, cost functions used in motion estimation for B-frames are simplified to reduce computational complexity. In one more embodiment, fractional pixel refinement in motion search for B-frames is simplified. In yet another embodiment, predictors used in motion estimation for a macro-block in a P-frame are selected from a B-frame that uses a same reference frame as the P-frame.

Claims (58)

1. A method of encoding image frames, the method comprising:

encoding, with one or more integrated circuits, each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;

disabling, with the one or more integrated circuits, a de-blocking filter for all encoded B-frames included in the plurality of encoded image frames;

for each non B-frame in the plurality of image frames, employing a filter with a first number of taps to perform sub-pixel interpolation to determine one or more motion vectors for the respective image frame; and

for each B-frame in the plurality of image frames, employing a filter with a second number of taps less than the first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame.

2. The method of claim 1 , wherein

each of the motion vectors specifies a displacement of a respective one of a plurality of macro-blocks in the respective image frame with respect to a reference frame.

3. The method of claim 1 , wherein the one or more integrated circuits are included in a video encoder, wherein the video encoder includes a first storage component external to the integrated circuits, and wherein bits representing a reconstructed image frame are written to the first storage component only if the reconstructed image frame is a non-B frame.

4. The method of claim 1 , wherein encoding each of the image frames includes:

encoding an image frame to form an encoded B-frame;

receiving pixel values representing a first macro-block contained in the image frame; and

determining, for the first macro-block, a cost associated with a potential best-match macro-block in a reference frame, the potential best-match macro-block in the reference frame potentially representing a best-match displacement of the first macro-block in the reference frame, wherein the cost contains a similarity indicator component and a motion vector cost component,

wherein determining the cost includes determining the motion-vector cost component based on motion vectors computed for each of a plurality of macro-blocks occurring earlier than the first macro-block in raster scan order in the image frame, as well as a predicted motion vector of an immediately preceding macro-block in raster scan order,

wherein the predicted motion vector represents an estimate of displacement of the immediately preceding macro-block in raster scan order with respect to the reference frame.

5. The method of claim 4 , wherein the cost contains only the similarity indicator component for evaluation of a motion vector predictor for motion estimation for a macro-block in the image frame.

6. The method of claim 5 , wherein the similarity indicator component is computed as a sum of absolute differences (SAD) of corresponding pixel values of pixels in the first macro-block and a potential best-match reference macro-block, and

wherein the B-frame is not used as a reference frame for encoding other frames.

7. A device comprising one or more integrated circuits configured to:

encode each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;

reconstruct each of the encoded image frames to generate corresponding reconstructed image frames, the reconstructed image frames including reconstructed B-frames and reconstructed non B-frames;

disable a de-blocking filter for all reconstructed B-frames included in the reconstructed image frames;

enable the de-blocking filter for all reconstructed non B-frames included in the reconstructed image frames; and

for each of the plurality of image frames, selectively employ a first filter with a first number of taps or a second filter with a second number of taps to perform sub-pixel interpolation to determine one or more motion vectors for the respective image frame based on whether the respective image frame is a B-frame or a non B-frame.

8. The device of claim 7 , wherein the integrated circuits are further configured to selectively employ the first or second filter by:

for each non B-frame in the plurality of image frames, employing a filter with a first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame; and

for each B-frame in the plurality of image frames, employing a filter with a second number of taps less than the first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame.

9. The device of claim 7 , wherein

each of the motion vectors specifies a displacement of a respective one of a plurality of macro-blocks in the respective image frame with respect to a reference frame.

10. The device of claim 7 , wherein the one or more integrated circuits are included in a video encoder, wherein the video encoder includes a first storage component external to the integrated circuits, and wherein bits representing a reconstructed image frame are written to the first storage component only if the reconstructed image frame is a non-B frame.

11. The device of claim 7 , wherein the one or more integrated circuits are further configured to:

encode an image frame to form an encoded B-frame;

receive pixel values representing a first macro-block contained in the image frame; and

determine, for the first macro-block, a cost associated with a potential best-match macro-block in a reference frame, the potential best-match macro-block in the reference frame potentially representing a best-match displacement of the first macro-block in the reference frame, wherein the cost contains a similarity indicator component and a motion vector cost component,

wherein the one or more integrated circuits are further configured to determine the cost at least in part by determining the motion-vector cost component based on motion vectors computed for each of a plurality of macro-blocks occurring earlier than the first macro-block in raster scan order in the image frame, as well as a predicted motion vector of an immediately preceding macro-block in raster scan order,

wherein the predicted motion vector represents an estimate of displacement of the immediately preceding macro-block in raster scan order with respect to the reference frame.

12. The device of claim 11 , wherein the cost contains only the similarity indicator component for evaluation of a motion vector predictor for motion estimation for a macro-block in the image frame.

13. The device of claim 12 ,

wherein the similarity indicator component is computed as a sum of absolute differences (SAD) of corresponding pixel values of pixels in the first macro-block and a potential best-match reference macro-block, and

wherein the B-frame is not used as a reference frame for encoding other frames.

14. A machine readable medium comprising instructions that, when executed by a video encoder, cause the video encoder to:

encode each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;

reconstruct each of the encoded image frames to generate corresponding reconstructed image frames, the reconstructed image frames including reconstructed B-frames and reconstructed non B-frames;

disable a de-blocking filter for all reconstructed B-frames included in the reconstructed image frames; and

enable the de-blocking filter for all reconstructed non B-frames included in the reconstructed image frames.

15. The machine readable medium of claim 14 , wherein the instructions further cause the video encoder to:

for each of the plurality of image frames, selectively employ a first filter with a first number of taps or a second filter with a second number of taps to perform sub-pixel interpolation to determine one or more motion vectors for the respective image frame based on whether the respective image frame is a B-frame or a non B-frame,

wherein each of the motion vectors specifies a displacement of a respective one of a plurality of macro-blocks in the respective image frame with respect to a reference frame.

16. The machine readable medium of claim 14 , wherein the video encoder includes a first storage component, and wherein bits representing a reconstructed image frame are written to the first storage component only if the reconstructed image frame is a non-B frame.

17. The machine readable medium of claim 14 , wherein the instructions that cause the video encoder to encode each of the image frames include instructions that cause the video encoder to:

encode an image frame to form an encoded B-frame;

receive pixel values representing a first macro-block contained in the image frame; and

determine, for the first macro-block, a cost associated with a potential best-match macro-block in a reference frame, the potential best-match macro-block in the reference frame potentially representing a best-match displacement of the first macro-block in the reference frame, wherein the cost contains a similarity indicator component and a motion vector cost component,

wherein the instructions that cause the video encoder to determine the cost include instructions that cause the video encoder to determine the motion-vector cost component based on motion vectors computed for each of a plurality of macro-blocks occurring earlier than the first macro-block in raster scan order in the image frame, as well as a predicted motion vector of an immediately preceding macro-block in raster scan order,

wherein the predicted motion vector represents an estimate of displacement of the immediately preceding macro-block in raster scan order with respect to the reference frame.

18. The machine readable medium of claim 17 , wherein the cost contains only the similarity indicator component for evaluation of a motion vector predictor for motion estimation for a macro-block in the image frame.

19. The machine readable medium of claim 18 ,

wherein the similarity indicator component is computed as a sum of absolute differences (SAD) of corresponding pixel values of pixels in the first macro-block and a potential best-match reference macro-block, and

wherein the B-frame is not used as a reference frame for encoding other frames.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 7, 2010
From: NAGORI, SOYEB; KUDANA, ARUN SHANKAR; SWAMI, PRAMOD KUMAR
To: TEXAS INSTRUMENTS INCORPORATED
Reel/Frame 024947/0498 →
Continuity (2)
Provisional Application 61176305 · May 7, 2009
Related Publication 20100284464A1 · Nov 11, 2010