Method and apparatus for periodic structure handling for motion compensation
View Patent ↗A motion compensated picture rate converter for determining a dominant motion vector for a block appearing in two images includes a high-pass filter and a low-pass filter, transform calculators responsive to the filters for performing transforms on at least two images to produce a frequency-domain representation of the images, estimating calculators for estimating a plurality of motion vectors based on the frequency-domain representations, and a periodic structure detection and elimination module responsive to the transform calculators and the estimating calculators for identifying a period based on the frequency-domain representation of the images and for selecting a dominant motion vector based on the estimated motion vectors and the identified period. A method of operation is also disclosed.
1. A method for video processing, the method comprising:
determining a pitch of a repeating pattern within a first frame;
determining, for a particular block of pixels within the first frame, a candidate motion vector defining object motion between the first frame and a second frame, wherein the candidate motion vector is determined as an average of motion vectors from all blocks that neighbor the particular block;
determining, for the particular block of pixels within the first frame, a first estimated motion vector defining object motion between the first frame and the second frame, wherein the first estimated motion vector is determined based on data that is derived from the first frame and the second frame;
calculating a first difference between the candidate motion vector and the estimated motion vector;
comparing the first difference to the pitch;
eliminating the estimated motion vector based on the difference being within a threshold value from a positive integer multiple of the pitch;
determining, for the particular block of pixels within the first frame, a second estimated motion vector defining object motion between the first frame and the second frame;
calculating a second difference between the candidate motion vector and the second estimated motion vector;
comparing the second difference to the pitch; and
determining that the second estimated motion vector is a correct motion vector based on the difference being beyond the threshold value from the pitch; and
applying the second estimated motion vector to the first frame to generate an intermediate frame between the first frame and the second frame.
2. The method of claim 1 , wherein determining the pitch includes:
performing a Fourier transform on data of the first frame, and
selecting a peak of the Fourier transformed data to indicate the pitch.
3. The method of claim 2 , wherein selecting the peak comprises selecting the peak based on a magnitude of the peak.
4. The method of claim 2 , wherein selecting the peak comprises selecting the peak based on a Fourier coefficient, of the peak, exceeding a threshold.
5. The method of claim 1 , wherein determining the pitch includes:
calculating the pitch separately for each of multiple blocks that neighbor the particular block, and
selecting a pitch, from among the pitches of the neighboring blocks, to be designated as the pitch for the particular block.
6. The method of claim 1 , wherein the pitch is determined separately in the horizontal direction and vertical direction.
7. The method of claim 1 , wherein determining the candidate motion vector includes:
performing a Fourier transform on data of frame,
performing a Fourier transform on data of the second frame,
calculating a phase difference between the Fourier transformed data of the frame data and the Fourier transformed data of the second frame, and
performing an inverse Fourier transform on the phase difference.
8. The method of claim 1 wherein:
data, from which the candidate motion vector is determined, is low-pass filtered, and
data, from which the estimated motion vector is determined, is high-pass filtered.
9. The method of claim 8 , wherein the pitch is determined from the high-pass filtered data.
10. The method of claim 1 , wherein the first frame is a previous frame, and the second frame is a subsequent frame.
11. The method of claim 1 wherein the positive integer is one.
12. The method of claim 1 , wherein the threshold value is a positive function of the pitch.
13. A video processing system comprising:
a processor;
a memory: and
instructions stored in the memory and executable by the processor configured to:
determine a pitch of a repeating pattern within a first frame;
determine, for a particular block of pixels within the first frame, a candidate motion vector defining object motion between the first frame and a second frame, wherein the candidate motion vector is determined as an average of motion vectors from all blocks that neighbor the particular block;
determine, for the particular block of pixels within the first frame, a first estimated motion vector defining object motion between the first frame and the second frame, wherein the first estimated motion vector is determined based on data that is derived from the first frame and the second frame;
calculate a first difference between the candidate motion vector and the estimated motion vector;
compare the first difference to the pitch;
eliminate the estimated motion vector based on the difference being within a threshold value from a positive integer multiple of the pitch;
determine, for the particular block of pixels within the first frame, a second estimated motion vector defining object motion between the first frame and the second frame;
calculate a second difference between the candidate motion vector and the second estimated motion vector;
compare the second difference to the pitch; and
determine that the second estimated motion vector is a correct motion vector based on the difference being beyond the threshold value from the pitch; and
apply the second estimated motion vector to the first frame to generate an intermediate frame between the first frame and the second frame.
14. The video processing system of claim 13 , wherein the determining of the pitch includes:
calculating the pitch separately for each of multiple blocks that neighbor the particular block, and
selecting a pitch, from among the pitches of the neighboring blocks, to be designated as the pitch for the particular block.
15. The video processing system of claim 13 , wherein the pitch is determined separately in the horizontal direction and vertical direction.
16. The video processing system of claim 13 , wherein:
data, from which the candidate motion vector is determined, is low-pass filtered, and
data, from which the estimated motion vector is determined, is high-pass filtered.
17. The video processing system of claim 16 , wherein the pitch is determined from the high-pass filtered data.
18. The video processing system of claim 13 , wherein the first frame is a previous frame, and the second frame is a subsequent frame.
19. The video processing system of claim 13 , wherein the positive integer is one.
20. The video processing system of claim 13 , wherein the threshold value is a positive function of the pitch.