IP Library › Granted Patent US 11,956,460
Granted Patent B2
US 11,956,460 · App. 16/556,015 · Granted Apr 9, 2024

Selective template matching in video coding

Inventors: Wenhao Zhang (Beijing, CN); Deliang Fu (Beijing, CN); Chen Liu (Beijing, CN); Xiaobo Liu (Beijing, CN)
Assignee: HULU, LLC
H04N19/513H04N19/105H04N19/176H04N19/182
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,956,460
App. No.
16/556,015
Granted
Apr 9, 2024
Kind
B2
Abstract

In some embodiments, a method selects a first template of pixels that is outside of a first block and a second template of pixels that is outside of the first block. Then, the method selects a first region of pixels that are inside the first block and a second region of pixels that are inside the first block. A first weight is calculated based on the first template of pixels and the first region of pixels and a second weight is calculated based on the second template of pixels and the second region of pixels. The first weight and the second weight are used in a motion prediction search for a second block.

Claims (78)

1. A method comprising:

selecting, by a computing device, a first template of pixels that is outside of a first block and a second template of pixels that is outside of the first block, wherein the first template and the first block are in a same frame and the second template and the first block are in the same frame;

selecting, by the computing device, a first region of pixels that are inside the first block and a second region of pixels that are inside the first block, wherein the first template and first region are similar sizes of a height and a width in the same frame and the second template and the second region are similar sizes of a height and a width in the same frame;

calculating, by the computing device, a first weight based on the first template of pixels and the first region of pixels;

calculating, by the computing device, a second weight based on the second template of pixels and the second region of pixels;

applying, by the computing device, the first weight to a third template of pixels that is outside of a second block to generate a weighted third template of pixels and the second weight to a fourth template of pixels that is outside of the second block to generate a weighted fourth template of pixels; and

using, by the computing device, the weighted third template of pixels and the weighted fourth template of pixels in a motion prediction search to determine a reference block for a block being coded in a different frame than the reference block.

2. The method of claim 1 , wherein:

the first template of pixels includes pixels to a left of the first block, and

the second template of pixels includes pixels to a top of the first block.

3. The method of claim 1 , wherein:

the first weight is associated with the first template and the second weight is associated with the second template,

the first template of pixels includes pixels that neighbor the pixels in the first region of pixels, and

the second template of pixels includes pixels that neighbor the pixels in the second region of pixels.

4. The method of claim 1 , wherein:

calculating the first weight comprises calculating a first difference between pixels in the first template of pixels and the first region of pixels; and

calculating the second weight comprises calculating a second difference between pixels in the second template of pixels and the second region of pixels.

5. The method of claim 4 , wherein:

calculating the first weight and the second weight comprises calculating a weighted distribution of the first difference and the second difference.

6. The method of claim 1 , further comprising:

inserting the first weight and the second weight in an encoded bitstream.

7. The method of claim 6 , wherein:

the first weight and the second weight are used to select the second block to use as the reference block for coding the first block.

8. The method of claim 1 , wherein:

the first block is in a first frame of a video,

the second block is in a second frame of the video, and

the first block is being coded.

9. The method of claim 1 , further comprising:

searching for a third block to use as the reference block for the second block, wherein the second block is being coded.

10. The method of claim 9 , wherein searching for the third block to use as the reference comprises:

determining a first template of pixels for the third block;

determining a second template of pixels for the third block;

applying the first weight to the first template of pixels for the third block to generate a weighted first template of pixels for the third block; and

applying the second weight to the second template of pixels for the third block to generate a weighted second template of pixels for the third block.

11. The method of claim 10 , wherein searching for the third block to use as the reference block comprises:

comparing the weighted first template of pixels for the second block to the weighted first template of pixels for the third block and comparing the weighted second template of pixels for the second block to the weighted second template of pixels for the third block to determine whether the second block is similar to the third block.

12. The method of claim 10 , wherein searching for the third block to use as the reference block comprises:

applying the first weight to a first template of pixels for a plurality of third blocks to generate a weighted first template of pixels for the plurality of third blocks;

applying the second weight to a second template of pixels for the plurality of third blocks to generate a weighted second template of pixels for the plurality of third blocks;

comparing the weighted first template of pixels for the plurality of third blocks to the weighted first template of pixels for the second block and comparing the weighted second template of pixels for the plurality of third blocks to the weighted second template of pixels for the second block; and

selecting one of the plurality of third blocks based on the comparing.

13. The method of claim 12 , wherein:

the plurality of third blocks are included in a search area in a same frame as the first block.

14. The method of claim 9 , wherein:

the second block is being decoded,

the first block is signaled as a reference block in an encoded bitstream, and

the third block is searched for as a new reference block to use to decode the second block.

15. The method of claim 9 , wherein:

the second block is in a first frame, and

the first block and the third block are in a second frame.

16. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be operable for:

selecting a first template of pixels that is outside of a first block and a second template of pixels that is outside of the first block, wherein the first template and the first block are in a same frame and the second template and the first block are in the same frame;

selecting a first region of pixels that are inside the first block and a second region of pixels that are inside the first block, wherein the first template and first region are similar sizes of a height and a width in the same frame and the second template and the second region are similar sizes of a height and a width in the same frame;

calculating a first weight based on the first template of pixels and the first region of pixels;

calculating a second weight based on the second template of pixels and the second region of pixels;

applying the first weight to a third template of pixels that is outside of a second block to generate a weighted third template of pixels and the second weight to a fourth template of pixels that is outside of the second block to generate a weighted fourth template of pixels; and

using the weighted third template of pixels and the weighted fourth template of pixels in a motion prediction search to determine a reference block for a block being coded in a different frame than the reference block.

17. The non-transitory computer-readable storage medium of claim 16 , further comprising:

searching for a third block to use as the reference block for the second block, wherein the second block is being coded.

18. The non-transitory computer-readable storage medium of claim 17 , wherein searching for the third block to use as the reference block comprises:

determining a first template of pixels for the third block;

determining a second template of pixels for the third block;

applying the first weight to the first template of pixels for the third block to generate a weighted first template of pixels for the third block; and

applying the second weight to the second template of pixels for the third block to generate a weighted second template of pixels for the third block.

19. The non-transitory computer-readable storage medium of claim 18 , wherein searching for the third block to use as the reference block comprises:

comparing the weighted first template of pixels for the second block to the weighted first template of pixels for the third block and comparing the weighted second template of pixels for the second block to the weighted second template of pixels for the third block to determine whether the second block is similar to the third block.

20. An apparatus comprising:

one or more computer processors; and

a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be operable for:

selecting a first template of pixels that is outside of a first block and a second template of pixels that is outside of the first block, wherein the first template and the first block are in a same frame and the second template and the first block are in the same frame;

selecting a first region of pixels that are inside the first block and a second region of pixels that are inside the first block, wherein the first template and first region are similar sizes of a height and a width in the same frame and the second template and the second region are similar sizes of a height and a width in the same frame;

calculating a first weight based on the first template of pixels and the first region of pixels;

calculating a second weight based on the second template of pixels and the second region of pixels;

applying the first weight to a third template of pixels that is outside of a second block to generate a weighted third template of pixels and the second weight to a fourth template of pixels that is outside of the second block to generate a weighted fourth template of pixels; and

using the weighted third template of pixels and the weighted fourth template of pixels in a motion prediction search to determine a reference block for a block being coded in a different frame than the reference block.

21. The method of claim 1 , wherein;

calculating the first weight based on the first template of pixels and the first region of pixels comprises comparing pixels in a same position in the first template of pixels and the first region of pixels; and

calculating the second weight based on the second template of pixels and the second region of pixels comprises comparing pixels in a same position in the second template of pixels and the second region of pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2019
From: ZHANG, WENHAO; FU, DELIANG; LIU, CHEN; LIU, XIAOBO
To: HULU, LLC.
Reel/Frame 050777/0194 →
Continuity (2)
Provisional Application 62726125 · Aug 31, 2018
Related Publication 20200077110A1 · Mar 5, 2020