IP Library Granted Patent US 10,602,151
Granted Patent B1
US 10,602,151 · App. 13/249,832 · Granted Mar 24, 2020

Estimated macroblock distortion co-optimization

Inventor: Jeffrey K. Lassahn (Portland, OR)
Assignee: AMAZON TECHNOLOGIES, INC.
H04N19/136H04N19/137H04N19/139H04N19/147H04N19/154H04N19/51H04N19/513
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,602,151
App. No.
13/249,832
Granted
Mar 24, 2020
Kind
B1
Abstract

An apparatus including a first module and a second module. The first module may be configured to generate one or more values based upon an analysis of one or more samples of a first frame. The second module may be configured to encode one or more samples of a second frame taking into account the one or more values generated by the first module. The one or more values generally represent a measure of an effect on the one or more samples of the first frame of encoding decisions made during encoding of the one or more samples of the second frame.

Claims (39)

1. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive a plurality of frames of a video sequence;

pre-encode said plurality of frames of said video sequence and analyze the pre-encoded frames to estimate how much distortion caused by encoding decisions in a reference frame persists in one or more subsequent frames, wherein the reference frame is a frame in the received plurality of frames;

generate one or more values for the reference frame based upon the analysis of the pre-encoded frames, wherein the one or more values are representative of distortions persisting from the reference frame into the one or more subsequent frames;

receive said plurality of frames of said video sequence and said one or more values for the reference frame;

determine a rate distortion value, based at least in part on the one or more values generated, representing estimated distortion to the one or more subsequent frames;

select a mode for encoding the reference frame based at least on the rate distortion value; and

encode one or more samples of the reference frame based on the mode.

2. The non-transitory computer-readable storage medium including instructions according to claim 1 , wherein said one or more values for the reference frame are based upon analysis of one or more macroblocks of the reference frame.

3. The non-transitory computer-readable storage medium including instructions according to claim 1 , wherein said pre-encoding and analysis comprises an approximated motion estimation.

4. The non-transitory computer-readable storage medium including instructions according to claim 1 , wherein said pre-encoding and analysis comprises an approximated psychovisual optimization.

5. The non-transitory computer-readable storage medium including instructions according to claim 1 , wherein said one or more values represent a measure of an amount of data from the reference frame that is used in predicting said one or more subsequent frames.

6. The non-transitory computer-readable storage medium including instructions according to claim 5 , wherein said one or more values are set to a first value when a future macroblock is an exact copy of a reference macroblock, set to a second value when no data from a reference macroblock is used in any future frames or set proportional to an amount of data from a reference macroblock that is used in any future frames.

7. The non-transitory computer-readable storage medium of claim 1 including further instructions that, when executed by the at least one processor of the computing device, further cause the computing device, to modify an encoding cost calculation based upon said one or more values.

8. The non-transitory computer-readable storage medium of claim 7 including further instructions that, when executed by the at least one processor of the computing device, further cause the computing device, to modify a rate distortion cost calculation based upon said one or more values.

9. The non-transitory computer-readable storage medium of claim 1 including further instructions that, when executed by the at least one processor of the computing device, further cause the computing device, to modify a measure of perceptual distortion calculation based upon said one or more values.

10. The non-transitory computer-readable storage medium of claim 1 including further instructions that, when executed by the at least one processor of the computing device, further cause the computing device, to perform said analysis a number of frame periods prior to the encoding of the one or more samples.

11. A method of estimated distortion co-optimization comprising:

pre-encoding and analyzing a plurality of frames of a video sequence to estimate how much distortion caused by an encoding decision in a reference frame persists in one or more subsequent frames, wherein the pre-encoding approximates a subsequent encoding process, and wherein the reference frame is a frame in the received plurality of frames;

generating one or more values for the reference frame based upon the analysis of the pre-encoded frames, wherein the one or more values are representative of distortions persisting from the reference frames into the one or more subsequent frames;

determining a rate distortion value based at least in part on the one or more values associated with estimated distortion to the one or more subsequent frames;

selecting a mode for encoding the reference frame based at least on the rate distortion value; and

encoding one or more samples of the reference frame based on the mode selection.

12. The method according to claim 11 , wherein said one or more values for the reference frame are based upon analysis of one or more macroblocks of the reference frame.

13. The method according to claim 11 , wherein said pre-encoding and analyzing comprises an approximated motion estimation.

14. The method according to claim 11 , wherein said pre-encoding and analyzing comprises an approximated psychovisual optimization.

15. The method according to claim 11 , wherein said one or more values represent a measure of an amount of data from the reference frame that is used in predicting said one or more subsequent frames.

16. The method according to claim 15 , wherein said one or more values are set to a first value when a future macroblock is an exact copy of a reference macroblock, set to a second value when no data from a reference macroblock is used in any future frames, or set proportional to an amount of data from a reference macroblock that is used in any future frames.

17. The method according to claim 11 , wherein encoding said one or more samples of the reference frame comprises modifying an encoding cost calculation based upon said one or more values.

18. The method according to claim 17 , wherein encoding said one or more samples of the reference frame comprises at least one of modifying a rate distortion cost calculation based upon said one or more values and modifying a measure of perceptual distortion calculation based upon said one or more values.

19. The apparatus according to claim 1 , wherein said analysis and said encoding are performed using a graphics processing unit (GPU) and said one or more values are stored in an array on said graphics processing unit.

20. An apparatus comprising:

one or more processors; and

memory including instructions that, upon being executed by the one or more processors, cause the apparatus to:

pre-encode and analyze a plurality of frames of a video sequence to estimate how much distortion caused by encoding decisions in a reference frame persists in one or more subsequent frames, wherein the pre-encoding approximates a subsequent encoding process, and wherein the reference frame is a frame in the received plurality of frames;

generate one or more values for the reference frame based upon the analysis of the pre-encoded frames, wherein the one or more values are representative of distortions persisting from the reference frame into the one or more subsequent frames;

determine a rate distortion value based at least in part on the one or more values associated with estimated distortion to the one or more subsequent frames;

select a mode for encoding the reference frame based at least on the rate distortion value; and

encode one or more macroblocks of the reference frame based on the mode selection.

Assignments (3)
BILL OF SALE Recorded Jan 12, 2016
From: ELEMENTAL TECHNOLOGIES LLC
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 037486/0985 →
CERTIFICATE OF CONVERSION Recorded Jan 12, 2016
From: ELEMENTAL TECHNOLOGIES, INC.
To: ELEMENTAL TECHNOLOGIES LLC
Reel/Frame 037486/0995 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2011
From: LASSAHN, JEFFREY K.
To: ELEMENTAL TECHNOLOGIES, INC.
Reel/Frame 027003/0834 →