IP Library Granted Patent US 9,596,481
Granted Patent B2
US 9,596,481 · App. 13/754,221 · Granted Mar 14, 2017

Apparatus and method for video data processing

Inventors: Sahar Alipour Kashi (Toronto, CA); Boris Ivanovic (North York, CA); Allen J. Porter (Sunderland, CA)
Assignee: ATI Technologies ULC
H04N19/521H04N19/117H04N19/423H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,596,481
App. No.
13/754,221
Granted
Mar 14, 2017
Kind
B2
Abstract

Methods and apparatus for facilitating processing a reference frame to produce an output frame. Motion vector data for a block of reference frame pels estimates the displacement of the reference frame pels from corresponding pels in a prior input frame. Comparison metrics are produced for a pel of the reference frame with respect to that pel and a plurality of neighboring reference frame pels A first comparison metric is based on a comparison with corresponding pels of a prior output frame that corresponds to the prior input frame as previously processed. A second comparison metric is based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying motion vector data to the pels of the prior output frame. A pel of the output frame that corresponds to the reference frame pel is determined using the first and second comparison metrics.

Claims (106)

1. An apparatus for facilitating processing image data of video frames comprising:

a processing component configured to process a reference frame to produce an output frame including:

a first input configured to receive motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

a second input configured to receive pel data for pels of a prior output frame corresponding to the prior input frame as previously processed;

the processing component configured to produce comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of the prior output frame including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

the processing component configured to determine a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the processing component is configured to use the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel and to use the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and

wherein the processing component is configured to produce the first and second blended pels and to use the first and second comparison metrics, a motion vector variance value, and a motion vector metric to determine a third gain for use in blending the first and second blended pels to produce a third blended pel upon which the output frame pel corresponding to the reference frame pel is based.

2. The apparatus of claim 1 wherein the processing component is configured to use the third blended pel as the output frame pel upon a condition that the third blended pel is in a range determined from the pel values of the reference frame pel and the plurality of neighboring reference frame pels.

3. An apparatus for facilitating processing image data of video frames comprising:

a processing component configured to process a reference frame to produce an output frame including:

a first input configured to receive motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

a second input configured to receive pel data for pels of a prior output frame corresponding to the prior input frame as previously processed;

the processing component configured to produce comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of the prior output frame including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

the processing component configured to determine a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the processing component is configured to use the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel and to use the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel;

wherein the processing component is configured to produce either the first blended pel or the second blended pel as a selected blended pel upon which the output frame pel corresponding to the reference frame pel is based; and

wherein the processing component is configured to use the selected blended pel as the output frame pel upon a condition that the selected blended pel is in a range determined from the pel values of the reference frame pel and the plurality of neighboring reference frame pels.

4. An apparatus for facilitating processing image data of video frames comprising:

a processing component configured to process a reference frame to produce an output frame including:

a first input configured to receive motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

a second input configured to receive pel data for pels of a prior output frame corresponding to the prior input frame as previously processed;

the processing component configured to produce comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of the prior output frame including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

the processing component configured to determine a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the processing component is configured to use the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel and to use the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and

wherein the processing component is configured to:

produce a third comparison metric for the reference frame pel based on a comparison of the reference frame pel and the plurality of neighboring reference frame pels with corresponding pels of a motion compensated prior input frame derived from applying the motion vector data for the reference frame block to the pels of the prior input frame; and

use the second and third comparison metrics in determining a noise level that is used in determining the first and second gains.

5. The apparatus of claim 4 wherein the processing component is configured to produce the first and second comparison metrics for the reference frame pel based on a comparison of a pel region defined by the reference frame pel and the plurality of neighboring pels that has fewer pels than the reference frame block.

6. The apparatus of claim 5 where the reference frame block is defined by at least an eight by eight pel array wherein the processing component is configured to produce the first and second comparison metrics for the reference frame pel based on a comparison of a pel region defined by a five by five pel array about the reference frame pel.

7. The apparatus of claim 1 wherein the processing component is configured to calculate the first and second comparison metrics for the output frame based on a sum of the absolute differences (SAD) comparative calculation.

8. A method of processing image data of a reference frame to produce an output frame comprising:

determining motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

producing comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of a prior output frame that corresponds to the prior input frame as previously processed including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

determining a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the determining a pel of the output frame includes:

using the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel; and

using the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and

wherein the determining a pel of the output frame includes:

producing the first and second blended pels;

using the first and second comparison metrics, a motion vector variance value, and a motion vector metric to determine a third gain; and

using the third gain in blending the first and second blended pels to produce a third blended pel.

9. The method of claim 8 wherein the determining a pel of the output frame includes:

using the third blended pel as the output frame pel upon a condition that the third blended pel is in a range determined from the pel values of the reference frame pel and the plurality of neighboring reference frame pels.

10. A method of processing image data of a reference frame to produce an output frame comprising:

determining motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

producing comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of a prior output frame that corresponds to the prior input frame as previously processed including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

determining a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the determining a pel of the output frame includes:

using the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel; and

using the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel;

wherein the determining a pel of the output frame includes producing either the first blended pel or the second blended pel as a selected blended pel upon which the output frame pel corresponding to the reference frame pel is based; and

wherein the determining a pel of the output frame includes:

using the selected blended pel as the output frame pel upon a condition that the selected blended pel is in a range determined from the pel values of the reference frame pel and the plurality of neighboring reference frame pels.

11. A method of processing image data of a reference frame to produce an output frame comprising:

determining motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

producing comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of a prior output frame that corresponds to the prior input frame as previously processed including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

determining a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the determining a pel of the output frame includes:

using the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel; and

using the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and wherein:

the producing comparison metrics for the reference frame pel includes producing a third comparison metric for the reference frame pel based on a comparison of the reference frame pel and the plurality of neighboring reference frame pels with corresponding pels of a motion compensated prior input frame derived from applying the motion vector data for the reference frame block to the pels of the prior input frame; and

the determining a pel of the output frame includes using the second and third comparison metrics in determining a noise level that is used in determining the first and second gains.

12. The method of claim 11 wherein the producing comparison metrics for the reference frame pel includes producing the first and second comparison metrics for the reference frame pel based on a comparison of a pel region defined by the reference frame pel and the plurality of neighboring pels that has fewer pels than the reference frame block.

13. The method of claim 12 where the reference frame block is defined by at least an eight by eight pel array wherein the producing comparison metrics for the reference frame pel includes producing the first and second comparison metrics for the reference frame pel based on a comparison of a pel region defined by a five by five pel array about the reference frame pel.

14. The method of claim 8 wherein the producing comparison metrics for the reference frame pel includes calculating the first and second comparison metrics for the output frame based on a sum of the absolute differences (SAD) comparative calculation.

15. A non-transitory computer-readable storage medium storing a set of instructions for execution by a processor to facilitate processing of video image data of a reference frame to produce an output frame that is adapted to:

determine motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

produce comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of a prior output frame that corresponds to the prior input frame as previously processed including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

determine a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the determine a pel of the output frame includes:

using the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel; and

using the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and

wherein the determine a pel of the output frame includes:

produce the first and second blended pels;

using the first and second comparison metrics, a motion vector variance value, and a motion vector metric to determine a third gain; and

using the third gain in blending the first and second blended pels to produce a third blended pel.

16. A non-transitory computer-readable storage medium storing a set of instructions for execution by one or more processors to facilitate manufacture of circuitry within an integrated circuit that is adapted to:

process a reference frame to produce an output frame including:

a first input configured to receive motion vector data for a block of pels of the reference frame that estimates the displacement of the reference frame pels from corresponding pels in a prior input frame; and

a second input configured to receive pel data for pels of a prior output frame corresponding to the prior input frame as previously processed;

produce comparison metrics for a reference frame pel of the reference frame block based on a comparison of the reference frame pel and a plurality of neighboring reference frame pels with pels of the prior output frame including:

a first comparison metric based on a comparison with corresponding pels of the prior output frame; and

a second comparison metric based on a comparison with corresponding pels of a motion compensated prior output frame derived from applying the motion vector data for the reference frame block to the pels of the prior output frame; and

determine a pel of the output frame that corresponds to the reference frame pel using the first and second comparison metrics;

wherein the determining a pel of the output frame includes:

using the first comparison metric to determine a first gain for use in blending the reference frame pel with a corresponding pel of the prior output frame to produce a first blended pel; and

using the second comparison metric to determine a second gain for use in blending the reference frame pel with a corresponding pel of the motion compensated prior output frame to produce a second blended pel; and

wherein:

the producing comparison metrics for the reference frame pel includes producing a third comparison metric for the reference frame pel based on a comparison of the reference frame pel and the plurality of neighboring reference frame pels with corresponding pels of a motion compensated prior input frame derived from applying the motion vector data for the reference frame block to the pels of the prior input frame; and

the determining a pel of the output frame includes using the second and third comparison metrics in determining a noise level that is used in determining the first and second gains.

17. The computer-readable storage medium of claim 16 , wherein the instructions are hardware description language (HDL) instructions used for the manufacture of a device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 21, 2013
From: KASHI, SAHAR ALIPOUR; IVANOVIC, BORIS; PORTER, ALLEN J.
To: ATI TECHNOLOGIES ULC
Reel/Frame 030059/0810 →
Continuity (1)
Related Publication 20140211855A1 · Jul 31, 2014