Granted Patent B2

US 12,238,335 · App. 18/302,625 · Granted Feb 25, 2025

Inventors: Yongmao Tang (Shanghai, CN); Jianjun Chen (Shanghai, CN); Junan Chen (Jiangsu, CN); Yonghai Wu (Shanghai, CN); Zejun Hu (Jiangsu, CN); Wei Feng (Shanghai, CN)

Assignee: NVIDIA Corporation

H04N19/61H04N19/105H04N19/109H04N19/122H04N19/139H04N19/156H04N19/172H04N19/176H04N19/182H04N19/423

Abstract

Claims (83)

1. A method comprising:

obtaining a first plurality of motion vectors of a first resolution;

generating a second plurality of motion vectors of a second resolution based on the first plurality of motion vectors of the first resolution;

calculating, for each motion vector of the second plurality of motion vectors, a first cost of the motion vector using a first cost function of a first size;

selecting a subset of motion vectors of the second plurality of motion vectors based on the first cost of each motion vector;

calculating, for each motion vector of the subset of motion vectors, a second cost using a second cost function of a second size;

generating a plurality of combined motion vectors based on the subset of motion vectors;

calculating, for each motion vector of the plurality of combined motion vectors, a third cost using the second cost function of the second size;

selecting, based on the cost of each motion vector of the subset of motion vectors and the cost of each motion vector of the plurality of combined motion vectors, a final motion vector; and

generating, based on the final motion vector, a block of predicted pixels that approximates a block of source pixels of an image frame.

2. The method of claim 1 , wherein calculating the first cost of the motion vector comprises, for at least a first motion vector of the second plurality of motion vectors:

obtaining pixel values of a corresponding reconstructed reference frame;

transforming the pixel values based on the first motion vector;

calculating a difference between the transformed pixel values and corresponding source pixel values; and

calculating the first cost using the first cost function of the first size based on the calculated difference.

3. The method of claim 1 , wherein calculating the first cost of the motion vector comprises, for at least a first motion vector of the second plurality of motion vectors, retrieving, from a buffer, the first cost of the first motion vector of the second plurality of motion vectors.

4. The method of claim 1 , wherein calculating the third cost comprises, for at least a first combined motion vector of the plurality of combined motion vectors:

retrieving, from a buffer, predicted pixel values corresponding to a first motion vector of the first combined motion vector;

obtaining pixel values of a reconstructed reference frame corresponding to a second motion vector of the first combined motion vector;

transforming the pixel values of the reconstructed reference frame based on the second motion vector;

calculating average pixel values based on the predicted pixel values and the transformed pixel values;

calculating a difference between the average pixel values and corresponding source pixel values; and

calculating the third cost using the second cost function of the second size based on the calculated difference.

5. The method of claim 1 , wherein the first resolution is an integer-pixel resolution and wherein the second resolution comprises at least one of a half-pixel resolution or a quarter-pixel resolution.

6. The method of claim 1 , wherein the first cost function of the first size is a 4×4 sum of absolute Hadamard-transformed difference function.

7. The method of claim 1 , wherein the second cost function of the second size is an 8×8 sum of absolute Hadamard-transformed difference function.

8. A system comprising:

a memory device; and

one or more circuit groups communicatively coupled to the memory device, the one or more circuit groups to:

obtain a first plurality of motion vectors of a first resolution;

generate a second plurality of motion vectors of a second resolution based on the first plurality of motion vectors of the first resolution;

calculate, for each motion vector of the second plurality of motion vectors, a first cost of the motion vector using a first cost function of a first size;

select a subset of motion vectors of the second plurality of motion vectors based on the first cost of each motion vector;

calculate, for each motion vector of the subset of motion vectors, a second cost using a second cost function of a second size;

generate a plurality of combined motion vectors based on the subset of motion vectors;

calculate, for each motion vector of the plurality of combined motion vectors, a third cost using the second cost function of the second size;

select, based on the cost of each motion vector of the subset of motion vectors and the cost of each motion vector of the plurality of combined motion vectors, a final motion vector; and

generate, based on the final motion vector, a block of predicted pixels that approximates a block of source pixels of an image frame.

9. The system of claim 8 , wherein to calculate the first cost of the motion vector, the one or more circuit groups are further to, for at least a first motion vector of the second plurality of motion vectors:

obtain pixel values of a corresponding reconstructed reference frame;

transform the pixel values based on the first motion vector;

calculate a difference between the transformed pixel values and corresponding source pixel values; and

calculate the first cost using the first cost function of the first size based on the calculated difference.

10. The system of claim 8 , wherein to calculate the first cost of the motion vector, the one or more circuit groups are further to, for at least a first motion vector of the second plurality of motion vectors, retrieve, from a buffer, the first cost of the first motion vector of the second plurality of motion vectors.

11. The system of claim 8 , wherein to calculate the third cost, the one or more circuit groups are further to, for at least a first combined motion vector of the plurality of combined motion vectors:

retrieve, from a buffer, predicted pixel values corresponding to a first motion vector of the first combined motion vector;

obtain pixel values of a reconstructed reference frame corresponding to a second motion vector of the first combined motion vector;

transform the pixel values of the reconstructed reference frame based on the second motion vector;

calculate average pixel values based on the predicted pixel values and the transformed pixel values;

calculate a difference between the average pixel values and corresponding source pixel values; and

calculate the third cost using the second cost function of the second size based on the calculated difference.

12. The system of claim 8 , wherein the first resolution is an integer-pixel resolution and wherein the second resolution comprises at least one of a half-pixel resolution or a quarter-pixel resolution.

13. The system of claim 8 , wherein the first cost function of the first size is a 4×4 sum of absolute Hadamard-transformed difference function.

14. The system of claim 8 , wherein the second cost function of the second size is an 8×8 sum of absolute Hadamard-transformed difference function.

15. A system comprising:

a memory device; and

one or more circuit groups communicatively coupled to the memory device, the one or more circuit groups comprising:

a first circuit group to:

obtain a first plurality of motion vectors of a first resolution;

generate a second plurality of motion vectors of a second resolution based on the first plurality of motion vectors of the first resolution;

calculate, for each motion vector of the second plurality of motion vectors, a first cost of the motion vector using a first cost function of a first size; and

select a subset of motion vectors of the second plurality of motion vectors based on the first cost of each motion vector; and

a second circuit group communicatively coupled to the first circuit group, the second circuit group to:

calculate, for each motion vector of the subset of motion vectors, a second cost using a second cost function of a second size;

generate a plurality of combined motion vectors based on the subset of motion vectors;

calculate, for each motion vector of the plurality of combined motion vectors, a third cost using the second cost function of the second size;

select, based on the cost of each motion vector of the subset of motion vectors and the cost of each motion vector of the plurality of combined motion vectors, a final motion vector; and

generate, based on the final motion vector, a block of predicted pixels that approximates a block of source pixels of an image frame.

16. The system of claim 15 , wherein to calculate the first cost of the motion vector, the first circuit group is further to, for at least a first motion vector of the second plurality of motion vectors:

obtain pixel values of a corresponding reconstructed reference frame;

transform the pixel values based on the first motion vector;

calculate a difference between the transformed pixel values and corresponding source pixel values; and

calculate the first cost using the first cost function of the first size based on the calculated difference.

17. The system of claim 15 , wherein to calculate the first cost of the motion vector, the first circuit group is further to, for at least a first motion vector of the second plurality of motion vectors, retrieve, from a buffer, the first cost of the first motion vector of the second plurality of motion vectors.

18. The system of claim 15 , wherein to calculate the third cost, the second circuit group is further to, for at least a first combined motion vector of the plurality of combined motion vectors:

retrieve, from a buffer, predicted pixel values corresponding to a first motion vector of the first combined motion vector;

obtain pixel values of a reconstructed reference frame corresponding to a second motion vector of the first combined motion vector;

transform the pixel values of the reconstructed reference frame based on the second motion vector;

calculate average pixel values based on the predicted pixel values and the transformed pixel values;

calculate a difference between the average pixel values and corresponding source pixel values; and

calculate the third cost using the second cost function of the second size based on the calculated difference.

19. The system of claim 15 , wherein the first resolution is an integer-pixel resolution and wherein the second resolution comprises at least one of a half-pixel resolution or a quarter-pixel resolution.

20. The system of claim 15 , wherein the second cost function of the second size is an 8×8 sum of absolute Hadamard-transformed difference function.

Assignments (1)

Continuity (1)

References Cited (85)