IP Library › Granted Patent US 11,533,471
Granted Patent B2
US 11,533,471 · App. 17/252,292 · Granted Dec 20, 2022

Image processing apparatus and image processing method

Inventor: Takeshi Tsukuba (Tokyo, JP)
Assignee: SONY CORPORATION
H04N19/105G06F7/24H04N19/132H04N19/137H04N19/159H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,533,471
App. No.
17/252,292
Granted
Dec 20, 2022
Kind
B2
Abstract

An image processing apparatus and an image processing method make it possible to suppress increase of the load of template matching. For each of motion vector candidates of accuracy rougher than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a reference block is performed to derive a cost, and elements of the motion vector candidate list are sorted on the basis of the derived costs of the motion vector candidates. Alternatively, for each of motion vector candidates of accuracy rougher than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a search point is performed to derive a cost, and a modification motion vector candidate is derived on the basis of the derived costs of the motion vector candidates.

Claims (46)

1. An image processing apparatus comprising:

processing circuitry configured to

perform, for each of motion vector candidates of lower accuracy than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a reference block to derive a cost;

sort elements of the motion vector candidate list on a basis of the costs of the motion vector candidates derived by the processing circuitry; and

perform template matching between the template of the current block having a sampling interval according to a size of the current block and the template of the reference block having a sampling interval according to the size of the current block to derive the cost.

2. The image processing apparatus according to claim 1 , the processing circuitry is further configured to:

derive a motion vector candidate list of integer pixel accuracy corresponding to a motion vector candidate list of decimal pixel accuracy, wherein

perform, for each of motion vector candidates in the motion vector candidate list of the integer pixel accuracy derived by the processing circuitry, template matching between the template of the current block and the template of the reference block to derive a cost, and

sort elements of the motion vector candidate list of the decimal pixel accuracy on a basis of the costs of the motion vector candidates derived by the processing circuitry.

3. The image processing apparatus according to claim 1 , wherein

the processing circuitry skips template matching for a motion vector candidate same as a motion vector candidate for which template matching has been performed.

4. The image processing apparatus according to claim 1 , wherein

the processing circuitry is configured to skip processing in a case where the current block is smaller than a predetermined threshold value.

5. The image processing apparatus according to claim 1 , wherein

the processing circuitry sets single prediction for motion vector candidates of multiple prediction and performs template matching between the template of the current block and the template of the reference block as a motion vector candidate of the single prediction to derive the cost.

6. The image processing apparatus according to claim 1 , wherein the processing circuitry is further configured to:

perform inter prediction to generate a prediction image and encode a motion vector to be used for generation of the prediction image by use of the motion vector candidate list in which the elements are sorted by the sorting section.

7. The image processing apparatus according to claim 1 , wherein the processing circuitry is further configured to:

restore a motion vector to be used for generation of a prediction image by use of the motion vector candidate list in which the elements are sorted by the sorting section.

8. An image processing method comprising:

performing, for each of motion vector candidates of lower accuracy than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a reference block to derive a cost;

sorting elements of the motion vector candidate list on a basis of the derived costs of the motion vector candidates; and

performing template matching between the template of the current block having a sampling interval according to a size of the current block and the template of the reference block having a sampling interval according to the size of the current block to derive the cost.

9. An image processing apparatus comprising:

processing circuitry configured to

perform, for each of motion vector candidates of lower accuracy than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a search point to derive a cost; and

derive a modification motion vector candidate on a basis of the costs of the motion vector candidates derived by the processing circuitry and update the motion vector candidate list by use of the derived modification motion vector candidate; and

perform template matching between the template of the current block of a size according to a position of the current block and the template of the search point of a size according to the position of the current block to derive the cost.

10. The image processing apparatus according to claim 9 , wherein the processing circuitry is further configured to:

derive a motion vector candidate list of integer pixel accuracy corresponding to a motion vector candidate list of decimal pixel accuracy, wherein

perform, for each of motion vector candidates in the motion vector candidate list of the integer pixel accuracy derived by the processing circuitry, template matching between the template of the current block and the template of the search point to derive a cost, and

derive the modification motion vector candidate on a basis of the costs of the motion vector candidates derived by the processing circuitry and update the motion vector candidate list of the decimal pixel accuracy by use of the derived modification motion vector candidate.

11. The image processing apparatus according to claim 9 , wherein

the processing circuitry skips template matching for a motion vector candidate same as a motion vector candidate for which template matching has been performed.

12. The image processing apparatus according to claim 9 , wherein

the processing circuitry skips processing in a case where the current block is smaller than a predetermined threshold value.

13. The image processing apparatus according to claim 9 , wherein

the processing circuitry sets single prediction for motion vector candidates of multiple prediction and performs template matching between the template of the current block and the template of the search point as a motion vector candidate of the single prediction to derive the cost.

14. The image processing apparatus according to claim 9 , wherein the processing circuitry is further configured to:

perform inter prediction to generate a prediction image and encode a motion vector to be used for generation of the prediction image by use of the motion vector candidate list updated by the update section using the modification motion vector candidate.

15. The image processing apparatus according to claim 9 , wherein the processing circuitry is further configured to:

restore a motion vector to be used for generation of a prediction image by use of the motion vector candidate list updated by the update section using the modification motion vector candidate.

16. An image processing method comprising:

performing, for each of motion vector candidates of lower accuracy than accuracy of a motion vector candidate list, template matching between a template of a current block and a template of a search point to derive a cost;

deriving a modification motion vector candidate on a basis of the derived costs of the motion vector candidates and updating the motion vector candidate list by use of the derived modification motion vector candidate; and

performing template matching between the template of the current block of a size according to a position of the current block and the template of the search point of a size according to the position of the current block to derive the cost.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2020
From: TSUKUBA, TAKESHI
To: SONY CORPORATION
Reel/Frame 054646/0346 →
Priority Claims (1)
JP JP2018-119063 · Jun 22, 2018 · national
Continuity (1)
Related Publication 20210266525A1 · Aug 26, 2021