IP Library Granted Patent US 8,675,736
Granted Patent B2
US 8,675,736 · App. 12/779,243 · Granted Mar 18, 2014

Motion vector processing

Inventors: Ai-Mei Huang (Taichung, TW); Chia-yuan Teng (San Diego, CA)
Assignee: QUALCOMM Incorporated
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,675,736
App. No.
12/779,243
Granted
Mar 18, 2014
Kind
B2
Abstract

This disclosure describes techniques for processing motion vectors such that the resulting motion vectors better correlate with the true motion of a video frame. In one example, the techniques may include comparing a block motion vector corresponding to a video block to a sub-block motion vector corresponding to a sub-block contained within the video block. The techniques may further include selecting one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on the comparison. Motion vectors that better correlate with true motion may be useful in applications such as motion compensated frame interpolation (MCI), moving object tracking, error concealment, or other video post-processing that requires the true motion information.

Claims (107)

1. A method of generating motion vectors in a video processing device, the method comprising:

determining a motion distance between a block motion vector and a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

comparing the motion distance to a first threshold;

selecting one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the motion distance is greater than the first threshold;

comparing the motion distance to a second threshold when the motion distance is not greater than the first threshold;

selecting the block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is greater than the second threshold; and

selecting the sub-block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is not greater than the second threshold.

2. The method of claim 1 , further comprising:

selecting one of the block motion vector and the sub-block motion vector based on the motion distance when the motion distance is not greater than the first threshold.

3. The method of claim 1 , further comprising:

performing motion compensated frame rate up conversion (FRUC) based on the spatially-estimated motion vector.

4. The method of claim 1 , further comprising:

comparing the spatially-estimated motion vector for the sub-block to a spatially-estimated motion vector for a neighboring sub-block; and

selecting one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block as a temporally-refined motion vector for the sub-block based on a comparison of the spatially-estimated motion vector for the sub-block to the spatially-estimated motion vector for the neighboring sub-block.

5. The method of claim 4 , wherein selecting one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block comprises:

selecting a motion vector from a set of motion vectors that minimizes a cost function, wherein the set of motion vectors includes the spatially-estimated motion vector for the sub-block and spatially-estimated motion vectors for neighboring sub-blocks.

6. The method of claim 4 , further comprising:

performing motion compensated frame rate up conversion (FRUC) based on the temporally-refined motion vector.

7. A method comprising:

comparing a block motion vector to a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

determining if the sub-block occupies a motion transition area of a video frame based on a comparison of the block motion vector to the sub-block motion vector; and

selecting one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the sub-block occupies the motion transition area of the video frame.

8. The method of claim 7 , further comprising:

selecting one of the block motion vector and the sub-block motion vector based on a motion distance between the block motion vector and the sub-block motion vector when the sub-block does not occupy the motion transition area of the video frame.

9. The method of claim 7 , wherein selecting one of the block motion vector and the sub-block motion vector based on the neighboring motion vectors comprises:

determining a first motion distance between the block motion vector and at least one spatially-estimated motion vector corresponding to a first neighboring sub-block;

determining a second motion distance between the sub-block motion vector and at least one spatially-estimated motion vector corresponding to a second neighboring sub-block;

comparing the first motion distance to the second motion distance; and

selecting one of the block motion vector and the sub-block motion vector based on a comparison of the first motion distance to the second motion distance.

10. A device for processing video signals, the device comprising:

a motion estimation module configured to:

determine a motion distance between a block motion vector and a sub-block motion vector,

compare the motion distance to a first threshold,

select one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the motion distance is greater than the first threshold,

compare the motion distance to a second threshold when the motion distance is not greater than the first threshold,

select the block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is greater than the second threshold, and

select the sub-block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is not greater than the second threshold,

wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector.

11. The device of claim 10 , wherein motion estimation module is further configured to select one of the block motion vector and the sub-block motion vector based on the motion distance when the motion distance is not greater than the first threshold.

12. The device of claim 10 , further comprising an interpolation module configured to perform motion compensated frame rate up conversion (FRUC) based on the spatially-estimated motion vector.

13. The device of claim 10 , wherein motion estimation module is further configured to compare the spatially-estimated motion vector for the sub-block to a spatially-estimated motion vector for a neighboring sub-block, and select one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block as a temporally-refined motion vector for the sub-block based on a comparison of the spatially-estimated motion vector for the sub-block to the spatially-estimated motion vector for the neighboring sub-block.

14. The device of claim 13 , wherein motion estimation module is further configured to select a motion vector from a set of motion vectors that minimizes a cost function, wherein the set of motion vectors includes the spatially-estimated motion vector for the sub-block and spatially-estimated motion vectors for neighboring sub-blocks.

15. The device of claim 13 , further comprising an interpolation module configured to perform motion compensated frame rate up conversion (FRUC) based on the temporally-refined motion vector.

16. The device of claim 10 , wherein the device comprises at least one of an integrated circuit, a microprocessor, or a wireless communication device.

17. A device comprising:

a motion estimation module configured to compare a block motion vector to a sub-block motion vector, determine if the sub-block occupies a motion transition area of a video frame based on a comparison of the block motion vector to the sub-block motion vector, and select one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the sub-block occupies the motion transition area of the video frame.

18. The device of claim 17 , wherein motion estimation module is further configured to select one of the block motion vector and the sub-block motion vector based on a motion distance between the block motion vector and the sub-block motion vector when the sub-block does not occupy the motion transition area of the video frame.

19. The device of claim 17 , wherein motion estimation module is further configured to determine a first motion distance between the block motion vector and at least one spatially-estimated motion vector corresponding to a first neighboring sub-block, determine a second motion distance between the sub-block motion vector and at least one spatially-estimated motion vector corresponding to a second neighboring sub-block, compare the first motion distance to the second motion distance, and select one of the block motion vector and the sub-block motion vector based on a comparison of the first motion distance to the second motion distance.

20. An apparatus for processing video signals, the apparatus comprising:

means for determining a motion distance between a block motion vector and a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

means for comparing the motion distance to a first threshold;

means for selecting one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the motion distance is greater than the first threshold;

means for comparing the motion distance to a second threshold when the motion distance is not greater than the first threshold;

means for selecting the block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is greater than the second threshold; and

means for selecting the sub-block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is not greater than the second threshold.

21. The apparatus of claim 20 , further comprising:

means for selecting one of the block motion vector and the sub-block motion vector based on the motion distance when the motion distance is not greater than the first threshold.

22. The apparatus of claim 20 , further comprising:

means for performing motion compensated frame rate up conversion (FRUC) based on the spatially-estimated motion vector.

23. The apparatus of claim 20 , further comprising:

means for comparing the spatially-estimated motion vector for the sub-block to a spatially-estimated motion vector for a neighboring sub-block; and

means for selecting one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block as a temporally-refined motion vector for the sub-block based on a comparison of the spatially-estimated motion vector for the sub-block to the spatially-estimated motion vector for the neighboring sub-block.

24. The apparatus of claim 23 , wherein the means for selecting one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block comprises:

means for selecting a motion vector from a set of motion vectors that minimizes a cost function, wherein the set of motion vectors includes the spatially-estimated motion vector for the sub-block and spatially-estimated motion vectors for neighboring sub-blocks.

25. The apparatus of claim 23 , further comprising:

means for performing motion compensated frame rate up conversion (FRUC) based on the temporally-refined motion vector.

26. The apparatus of claim 20 , wherein the apparatus comprises at least one of an integrated circuit, a microprocessor, or a wireless communication device.

27. An apparatus comprising:

means for comparing a block motion vector to a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

means for determining if the sub-block occupies a motion transition area of a video frame based on a comparison of the block motion vector to the sub-block motion vector; and

means for selecting one of the block motion vector and the sub-block motion vector based on neighboring motion vectors when the sub-block occupies the motion transition area of the video frame.

28. The apparatus of claim 27 , further comprising:

means for selecting one of the block motion vector and the sub-block motion vector based on a motion distance between the block motion vector and the sub-block motion vector when the sub-block does not occupy the motion transition area of the video frame.

29. The apparatus of claim 28 , wherein the means for selecting one of the block motion vector and the sub-block motion vector based on the neighboring motion vectors comprises:

means for determining a first motion distance between the block motion vector and at least one spatially-estimated motion vector corresponding to a first neighboring sub-block;

means for determining a second motion distance between the sub-block motion vector and at least one spatially-estimated motion vector corresponding to a second neighboring sub-block;

means for comparing the first motion distance to the second motion distance; and

means for selecting one of the block motion vector and the sub-block motion vector based on a comparison of the first motion distance to the second motion distance.

30. A non-transitory computer-readable storage medium storing computer-executable instructions that upon execution by one or more processors cause the one or more processors to:

determine a motion distance between a block motion vector and a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

compare the motion distance to a first threshold;

select one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on neighboring motion vectors when the motion distance is greater than the first threshold;

compare the motion distance to a second threshold when the motion distance is not greater than the first threshold;

select the block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is greater than the second threshold; and

select the sub-block motion vector as the spatially-estimated motion vector for the sub-block when the motion distance is not greater than the second threshold.

31. The non-transitory computer-readable storage medium of claim 30 , wherein the instructions further cause the one or more processors to:

select one of the block motion vector and the sub-block motion vector based on the motion distance when the motion distance is not greater than the first threshold.

32. The non-transitory computer-readable storage medium of claim 30 , wherein the instructions further cause the one or more processors to:

perform motion compensated frame rate up conversion (FRUC) based on the spatially-estimated motion vector.

33. The non-transitory computer-readable storage medium of claim 30 , wherein the instructions further cause the one or more processors to:

compare the spatially-estimated motion vector for the sub-block to a spatially-estimated motion vector for a neighboring sub-block; and

select one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block as a temporally-refined motion vector for the sub-block based on a comparison of the spatially-estimated motion vector for the sub-block to the spatially-estimated motion vector for the neighboring sub-block.

34. The non-transitory computer-readable storage medium of claim 33 , wherein the instructions that upon execution by the one or more processors cause the one or more processors to select one of the spatially-estimated motion vector for the sub-block and the spatially-estimated motion vector for the neighboring sub-block comprise instructions that upon execution by the one or more processors cause the one or more processors to:

select a motion vector from a set of motion vectors that minimizes a cost function, wherein the set of motion vectors includes the spatially-estimated motion vector for the sub-block and spatially-estimated motion vectors for neighboring sub-blocks.

35. The non-transitory computer-readable storage medium of claim 33 , wherein the instructions further cause the one or more processors to:

perform motion compensated frame rate up conversion (FRUC) based on the temporally-refined motion vector.

36. A non-transitory computer-readable storage medium storing computer-executable instructions that upon execution by one or more processors cause the one or more processors to:

compare a block motion vector to a sub-block motion vector, wherein the block motion vector corresponds to a video block that includes a sub-block corresponding to the sub-block motion vector;

determine if the sub-block occupies a motion transition area of a video frame based on a comparison of the block motion vector to the sub-block motion vector; and

select one of the block motion vector and the sub-block motion vector based on neighboring motion vectors when the sub-block occupies the motion transition area of the video frame.

37. The non-transitory computer-readable storage medium of claim 36 , wherein the instructions further cause the one or more processors to:

select one of the block motion vector and the sub-block motion vector based on a motion distance between the block motion vector and the sub-block motion vector when the sub-block does not occupy the motion transition area of the video frame.

38. The non-transitory computer-readable storage medium of claim 36 , wherein the instructions that upon execution by the one or more processors cause the one or more processors to select one of the block motion vector and the sub-block motion vector based on the neighboring motion vectors comprise instructions that upon execution by the one or more processors cause the one or more processors to:

determine a first motion distance between the block motion vector and at least one spatially-estimated motion vector corresponding to a first neighboring sub-block;

determine a second motion distance between the sub-block motion vector and at least one spatially-estimated motion vector corresponding to a second neighboring sub-block;

compare the first motion distance to the second motion distance; and

select one of the block motion vector and the sub-block motion vector based on a comparison of the first motion distance to the second motion distance.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2010
From: HUANG, AI-MEI; TENG, CHIA-YUAN
To: QUALCOMM INCORPORATED
Reel/Frame 024379/0458 →
Continuity (2)
Provisional Application 61178185 · May 14, 2009
Related Publication 20100290530A1 · Nov 18, 2010