IP Library › Granted Patent US 11,716,438
Granted Patent B2
US 11,716,438 · App. 17/216,633 · Granted Aug 1, 2023

Method for motion estimation, non-transitory computer-readable storage medium, and electronic device

Inventor: Wenting Shu (Guangdong, CN)
Assignee: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.
H04N5/145H04N19/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,716,438
App. No.
17/216,633
Granted
Aug 1, 2023
Kind
B2
Abstract

A method for motion estimation, a non-transitory computer-readable storage medium, and an electronic device are provided. The method includes the following. An error between a block to be matched and each of reference blocks is determined to obtain multiple errors. At least one candidate block is determined from the reference blocks according to the multiple errors. A random number range is determined according to a displacement between the block to be matched and each of the reference blocks. Random motion vectors are obtained by using a motion vector between the block to be matched and each of the at least one candidate block as a basic motion vector and random numbers within the random number range as increments. A target motion vector corresponding to the block to be matched is determined according to the random motion vectors.

Claims (76)

1. A method for motion estimation, comprising:

determining an error between a block to be matched and each of reference blocks to obtain a plurality of errors, wherein the block to be matched is an image block in a current frame, and the reference blocks are image blocks in a reference frame corresponding to the current frame;

determining at least one candidate block from the reference blocks according to the plurality of errors;

determining a random number range according to a displacement between the block to be matched and each of the reference blocks;

obtaining random motion vectors by using a motion vector between the block to be matched and each of the at least one candidate block as a basic motion vector and random numbers within the random number range as increments; and

determining a target motion vector corresponding to the block to be matched according to the random motion vectors.

2. The method of claim 1 , wherein the error between the block to be matched and any one of the reference blocks is determined according to one or more of following error terms: a sum of absolute differences (SAD), a gradient difference, a penalty value, or a Euclidean distance.

3. The method of claim 2 , wherein determining the gradient difference between the block to be matched and each of the reference blocks comprises:

calculating, for each of edge pixel points of the block to be matched in the current frame, a gradient between the edge pixel point and a corresponding adjacent external pixel point, and summing gradients calculated for the edge pixel points of the block to be matched to obtain a gradient of the block to be matched;

calculating, for each of edge pixel points of the reference block in the reference frame, a gradient between the edge pixel point and a corresponding adjacent external pixel point, and summing gradients calculated for the edge pixel points of the reference block to obtain a gradient of the reference block; and

obtaining the gradient difference between the block to be matched and the reference block by calculating an absolute value of a difference between the gradient of the block to be matched and the gradient of the reference block.

4. The method of claim 2 , further comprising:

on condition that the error is determined according to at least two error terms, determining the error between the block to be matched and any one of the reference blocks by performing weighted calculation on the at least two error terms.

5. The method of claim 2 , wherein determining the at least one candidate block from the reference blocks according to the plurality of errors comprises:

sorting the reference blocks in an ascending order of the plurality of errors; and

selecting the at least one candidate block starting from a first reference block among the reference blocks.

6. The method of claim 5 , wherein sorting the reference blocks in the ascending order of the plurality of errors comprises:

determining, according to a preset error term, an arrangement order of at least two reference blocks among the reference blocks, on condition that an error between the block to be matched and any one of the at least two reference blocks is the same as that between the block to be matched and another of the at least two reference blocks;

wherein the preset error term comprises any one of the following error terms or a combination of two or three of the following error terms: the SAD, the gradient difference, the penalty value, or the Euclidean distance.

7. The method of claim 1 , wherein determining the at least one candidate block from the reference blocks according to the plurality of errors comprises:

determining, according to at least two error thresholds, a range to which the error between the block to be matched and each of the reference blocks belongs, to obtain an error distribution, wherein the error distribution is associated with determining of the at least one candidate block; and

determining the at least one candidate block from the reference blocks according to the error distribution.

8. The method of claim 7 , wherein determining, according to the at least two error thresholds, the range to which the error between the block to be matched and each of the reference blocks belongs, to obtain the error distribution comprises:

for each of the reference blocks, determining a type of the reference block according to the at least two error thresholds; and

obtaining the error distribution according to the type of each of the reference blocks;

wherein the at least two error thresholds comprise a first error threshold and a second error threshold greater than the first error threshold, and determining the type of each of the reference blocks according to the at least two error thresholds comprises:

determining the reference block as a first-type reference block on condition that an error corresponding to the reference block is lower than the first error threshold;

determining the reference block as a second-type reference block on condition that the error corresponding to the reference block is between the first error threshold and the second error threshold; or

determining the reference block as a third-type reference block on condition that the error corresponding to the reference block is higher than the second error threshold.

9. The method of claim 8 , wherein determining the at least one candidate block from the reference blocks according to the error distribution comprises one of:

on condition that at least one first-type reference block is determined, determining the at least one candidate block from the at least one first-type reference block; or

on condition that no first-type reference block is determined and at least one second-type reference block is determined, determining the at least one candidate block from the at least one second-type reference block.

10. The method of claim 9 , wherein determining the at least one candidate block from the reference blocks according to the error distribution further comprises:

on condition that no first-type reference block and no second-type reference block are determined, determining the at least one candidate block from at least one third-type reference block.

11. The method of claim 7 , wherein determining, according to the at least two error thresholds, the range to which the error between the block to be matched and each of the reference blocks belongs, to obtain the error distribution comprises:

sorting the reference blocks in an ascending order of the plurality of errors, and selecting a preset number of reference blocks starting from a first reference block among the reference blocks; and

determining, according to the at least two error thresholds, the range to which the error between the block to be matched and each of the preset number of reference blocks belongs to obtain the error distribution.

12. The method of claim 1 , wherein determining the random number range according to the displacement between the block to be matched and each of the reference blocks comprises:

determining a displacement between the block to be matched and each of at least one reference block in a preset direction of the block to be matched to obtain a plurality of displacements, and determining a first displacement by averaging the plurality of displacements obtained for the at least one reference block in the preset direction of the block to be matched;

determining a displacement between the block to be matched and each of the at least one candidate block to obtain a plurality of displacements, and determining a second displacement by averaging the plurality of displacements obtained for the at least one candidate block; and

determining the random number range according to the first displacement and the second displacement.

13. The method of claim 12 , wherein determining the random number range according to the first displacement and the second displacement comprises:

calculating an average displacement of the first displacement and the second displacement; and

converting the average displacement to obtain the random number range.

14. The method of claim 1 , wherein using the motion vector between the block to be matched and each of the at least one candidate block as the basic motion vector comprises:

calculating, on condition that the at least one candidate block is embodied as a plurality of candidate blocks, a motion vector between the block to be matched and each of the plurality of candidate blocks to obtain a plurality of motion vectors; and

averaging the plurality of motion vectors to obtain the basic motion vector.

15. The method of claim 1 , wherein obtaining the random motion vectors by using the motion vector between the block to be matched and each of the at least one candidate block as the basic motion vector and the random numbers within the random number range as the increments comprises:

traversing the random number range in unit step size to obtain a plurality of sets of increments; and

using the motion vector between the block to be matched and each of the at least one candidate block as the basic motion vector, and obtaining the random motion vectors according to the basic motion vector and the plurality of sets of increments.

16. The method of claim 15 , wherein determining the target motion vector corresponding to the block to be matched according to the random motion vectors comprises:

obtaining, in the reference frame, a reference block corresponding to each of the random motion vectors, and calculating an error between the block to be matched and the reference block corresponding to each of the random motion vectors; and

determining a random motion vector corresponding to a minimum error as the target motion vector when errors converge.

17. An electronic device comprising:

at least one processor; and

a non-transitory computer readable storage, coupled to the at least one processor and storing at least one computer executable instruction which, when executed by the at least one processor, causes the at least one processor to:

determine an error between a block to be matched and each of reference blocks to obtain a plurality of errors, wherein the block to be matched is an image block in a current frame, and the reference blocks are image blocks in a reference frame corresponding to the current frame;

determine at least one candidate block from the reference blocks according to the plurality of errors;

determine a random number range according to a displacement between the block to be matched and each of the reference blocks;

obtain random motion vectors by using a motion vector between the block to be matched and each of the at least one candidate block as a basic motion vector and random numbers within the random number range as increments; and

determine a target motion vector corresponding to the block to be matched according to the random motion vectors.

18. The electronic device of claim 17 , wherein the at least one processor configured to determine the at least one candidate block from the reference blocks according to the plurality of errors is configured to:

determine, according to at least two error thresholds, a range to which the error between the block to be matched and each of the reference blocks belongs, to obtain an error distribution, wherein the error distribution is associated with determining of the at least one candidate block; and

determine the at least one candidate block from the reference blocks according to the error distribution.

19. The electronic device of claim 18 , wherein the at least one processor configured to determine, according to the at least two error thresholds, the range to which the error between the block to be matched and each of the reference blocks belongs, to obtain the error distribution is configured to:

for each of the reference blocks, determine a type of the reference block according to the at least two error thresholds; and

obtain the error distribution according to the type of each of the reference blocks; wherein the at least two error thresholds comprise a first error threshold and a second error threshold greater than the first error threshold, and the at least one processor configured to determine the type of each of the reference blocks according to the at least two error thresholds is configured to:

determine the reference block as a first-type reference block on condition that an error corresponding to the reference block is lower than the first error threshold;

determine the reference block as a second-type reference block on condition that the error corresponding to the reference block is between the first error threshold and the second error threshold; or

determine a reference block as a third-type reference block on condition that the error corresponding to the reference block is higher than the second error threshold.

20. A non-transitory computer-readable storage medium storing computer programs which, when executed by a processor, are operable with the processor to:

determine an error between a block to be matched and each of reference blocks to obtain a plurality of errors, wherein the block to be matched is an image block in a current frame, and the reference blocks are image blocks in a reference frame corresponding to the current frame;

determine at least one candidate block from the reference blocks according to the plurality of errors;

determine a random number range according to a displacement between the block to be matched and each of the reference blocks;

obtain random motion vectors by using a motion vector between the block to be matched and each of the at least one candidate block as a basic motion vector and random numbers within the random number range as increments; and

determine a target motion vector corresponding to the block to be matched according to the random motion vectors.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2021
From: SHU, WENTING
To: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.
Reel/Frame 055779/0413 →
Priority Claims (1)
CN 202010239649.4 · Mar 30, 2020 · national
Continuity (1)
Related Publication 20210306528A1 · Sep 30, 2021