IP Library Granted Patent US 11,558,637
Granted Patent B1
US 11,558,637 · App. 16/716,001 · Granted Jan 17, 2023

Unified search window to support multiple video encoding standards

Inventors: Xianliang Zha (El Dorado Hills, CA); Harikrishna Madadi Reddy (San Jose, CA); Junqiang Lan (Fremont, CA); Sujith Srinivasan (Cupertino, CA); Chung-Fu Lin (Menlo Park, CA); Guogang Hua (San Diego, CA)
Assignee: Meta Platforms, Inc.
H04N19/56H04N19/105H04N19/132H04N19/139H04N19/176H04N19/43
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,637
App. No.
16/716,001
Granted
Jan 17, 2023
Kind
B1
Abstract

A system comprises a memory storage configured to store at least a portion of a frame of a video and a hardware motion estimation search processing unit configured to perform at least a portion of a motion estimation search for the video for a plurality of different block sizes. The hardware motion estimation search processing unit is configured to perform the motion estimation search using a plurality of source sub-blocks of a first block size to determine a first type of comparison evaluation values for the first block size. A combination of values included in the first type of comparison evaluation values is utilized to determine at least one second type of comparison evaluation values for a second block size, wherein the second block size is larger than the first block size.

Claims (32)

1. A system comprising:

a memory storage configured to store at least a portion of a first frame and a second frame of a video; and

a plurality of hardware motion estimation search processing units configured to perform at least a portion of motion estimation searches for the video for a plurality of different frames and a plurality of different block sizes in parallel, including by being configured to perform the motion estimation searches for the plurality of different frames in parallel using a plurality of source sub-blocks of a first block size to determine a first type of comparison evaluation values for the first block size of a first video format and, separate from the determination of the first type of comparison evaluation values, calculate a combination of values included in the first type of comparison evaluation values as at least one second type of comparison evaluation value representing a motion estimate of a block of a second block size of a second video format different from the first video format as a part of a combined unified motion estimation search targeting multiple video formats during the same combined unified motion estimation search using a combined unified process that targets together each of the multiple video formats to determine together based on properties of the multiple video formats at least the first type of comparison evaluation values and the second type of comparison evaluation value, wherein the second block size is larger than the first block size, and wherein the hardware motion estimation search processing unit is configured to utilize at least a portion of the first type of comparison evaluation values for the first block size and the at least one second type of comparison evaluation value assigned to the block of the second block size as a common basis in encoding the video into the multiple video formats.

2. The system of claim 1 , wherein the first frame of the video and the second frame of the video are reference frames of the video.

3. The system of claim 2 , wherein the first frame of the video is a down-sampled version of an original video.

4. The system of claim 3 , wherein the down-sampled version of the original video was down-sampled by a factor of 16 or by a factor of 4.

5. The system of claim 1 , wherein the first block size is a 4×4 pixel block.

6. The system of claim 5 , wherein the second block size is a 4×8, 8×4, 8×8, 8×16, 16×8, or 16×16 pixel block.

7. The system of claim 1 , wherein the plurality of source sub-blocks are arranged as a column.

8. The system of claim 7 , wherein the column includes exactly four sub-blocks.

9. The system of claim 8 , wherein the plurality of source sub-blocks are a portion of a source frame block.

10. The system of claim 9 , wherein the source frame block is a 16×16 pixel block.

11. The system of claim 1 , wherein at least one of the hardware motion estimation search processing units is configured to determine the first type of comparison evaluation values including by being configured to utilize a sum of absolute differences measurement.

12. The system of claim 1 , wherein at least one of the hardware motion estimation search processing units is configured to determine the at least one second type of comparison evaluation value including by being configured to utilize a sum of absolute differences measurement.

13. The system of claim 1 , wherein the hardware motion estimation search processing units are configured to perform at least the portion of the motion estimation searches for the video at least in part concurrently for the plurality of different block sizes.

14. The system of claim 1 , wherein at least one of the hardware motion estimation search processing units is configured to determine the at least one second type of comparison evaluation value including by being configured to sum the combination of values included in the first type of comparison evaluation values.

15. A method, comprising:

storing at least a portion of a first frame and a second frame of a video; and

using a plurality of hardware motion estimation search processing units to perform at least a portion of motion estimation searches for the video for a plurality of different frames and a plurality of different block sizes in parallel, including by:

performing the motion estimation searches for the plurality of different frames in parallel using a plurality of source sub-blocks of a first block size to determine a first type of comparison evaluation values for the first block size of a first video format; and

separate from the determination of the first type of comparison evaluation values, calculating a combination of values included in the first type of comparison evaluation values as at least one second type of comparison evaluation value representing a motion estimate of a block of a second block size of a second video format different from the first video format as a part of a combined unified motion estimation search targeting multiple video formats during the same combined unified motion estimation search using a combined unified process that targets together each of the multiple video formats to determine together based on properties of the multiple video formats at least the first type of comparison evaluation values and the second type of comparison evaluation value;

wherein the second block size is larger than the first block size and wherein the hardware motion estimation search processing unit is configured to utilize at least a portion of the first type of comparison evaluation values for the first block size and the at least one second type of comparison evaluation value assigned to the block of the second block size as a common basis in encoding the video into the multiple video formats.

16. The method of claim 15 , wherein the video is a down-sampled version of an original video.

17. The method of claim 15 , wherein performing the motion estimation search using the plurality of source sub-blocks to determine the first type of comparison evaluation values includes utilizing a sum of absolute differences measurement.

18. The method of claim 15 , further comprising identifying a motion vector based on the at least one second type of comparison evaluation value.

19. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

storing at least a portion of a first frame and a second frame of a video; and

using a plurality of hardware motion estimation search processing units to perform at least a portion of motion estimation searches for the video for a plurality of different frames and a plurality of different block sizes in parallel, including by:

performing the motion estimation searches for the plurality of different frames in parallel using a plurality of source sub-blocks of a first block size to determine a first type of comparison evaluation values for the first block size of a first video format; and

separate from the determination of the first type of comparison evaluation values, calculating a combination of values included in the first type of comparison evaluation values as at least one second type of comparison evaluation value representing a motion estimate of a block of a second block size of a second video format different from the first video format as a part of a combined unified motion estimation search targeting multiple video formats during the same combined unified motion estimation search using a combined unified process that targets together each of the multiple video formats to determine together based on properties of the multiple video formats at least the first type of comparison evaluation values and the second type of comparison evaluation value;

wherein the second block size is larger than the first block size, and wherein the hardware motion estimation search processing unit is configured to utilize at least a portion of the first type of comparison evaluation values for the first block size and the at least one second type of comparison evaluation value assigned to the block of the second block size as a common basis in encoding the video into the multiple video formats.

20. The computer program product of claim 19 , wherein the first frame of the video and the second frame of the video are reference frames of the video.

Assignments (2)
CHANGE OF NAME Recorded Nov 19, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058214/0351 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2020
From: ZHA, XIANLIANG; REDDY, HARIKRISHNA MADADI; LAN, JUNQIANG; SRINIVASAN, SUJITH; LIN, CHUNG-FU; HUA, GUOGANG
To: FACEBOOK, INC.
Reel/Frame 051940/0616 →