IP Library Granted Patent US 11,909,993
Granted Patent B1
US 11,909,993 · App. 17/390,749 · Granted Feb 20, 2024

Fractional motion estimation engine with parallel code unit pipelines

Inventors: Kameswara Kishore Sriadibhatla (Dublin, CA); Yunqing Chen (Los Altos, CA); Junqiang Lan (Fremont, CA); Adrian Stafford Lewis (Mountain View, CA); Anil Muthiraparampil Sunil (Sunnyvale, CA)
Assignee: META PLATFORMS, INC.
H04N19/436H04N19/119H04N19/43H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,909,993
App. No.
17/390,749
Granted
Feb 20, 2024
Kind
B1
Abstract

A video encoder is disclosed. The video encoder comprises an integer level motion estimation hardware component configured to determine candidate integer level motion vectors for a video being encoded. The video encoder comprises a fractional motion estimation hardware component configured to receive the candidate integer level motion vectors from the integer level motion estimation hardware component and refine the candidate integer level motion vectors into candidate sub-pixel level motion vectors. The fractional motion estimation hardware component includes parallel pipelines configured to process coding units of a frame of the video in parallel across the parallel pipelines.

Claims (33)

1. A system, comprising:

an integer level motion estimation hardware component configured to determine candidate integer level motion vectors for a video being encoded; and

a fractional motion estimation hardware component configured to:

receive the candidate integer level motion vectors from the integer level motion estimation hardware component;

divide the candidate integer level motion vectors from the integer level motion estimation hardware component into a plurality of groups, wherein each group of the plurality of groups corresponds to a different single row of coding units of the frame of the video; and

process a group of the plurality of groups by one of a plurality of parallel row pipelines, wherein a parallel row pipeline is configured to process in parallel with other parallel row pipelines a different single row of coding units of the frame of the video, and wherein the plurality of parallel pipelines refine the candidate integer level motion vectors into candidate sub-pixel level motion vectors.

2. The system of claim 1 , wherein the fractional motion estimation hardware component is further configured to compute a fractional motion vector and send the fractional motion vector to an inter prediction module of a mode decision module.

3. The system of claim 1 , wherein a coding unit of the frame of the video comprises a largest coding unit that may be divided into smaller prediction units.

4. The system of claim 1 , wherein in the event of data dependencies in processing the coding units across different rows of coding units of the frame of the video, the processing of a certain coding unit in a particular row and column is performed after the processing of another coding unit in a previous row and a particular column is finished by a different parallel pipeline.

5. The system of claim 1 , wherein a parallel pipeline comprises parallel partition unit pipelines, and wherein a partition unit pipeline is configured to process in parallel with the other partition unit pipelines different partition units, and wherein the partition unit pipeline is configured to process a different set of partition units.

6. The system of claim 5 , wherein a first parallel partition unit pipeline is configured to process partition units with the smallest partition unit shapes, and wherein a second parallel partition unit pipeline is configured to process partition units with the remaining larger partition unit shapes.

7. The system of claim 5 , wherein a parallel partition unit pipeline comprises pipelined half pixel interpolation, quarter pixel interpolation, and one-eighth pixel interpolation for refining the candidate integer level motion vectors into the candidate sub-pixel level motion vectors.

8. The system of claim 7 , wherein nine positions are searched in a half-pixel refinement, and wherein nine positions are searched in a quarter-pixel refinement, and wherein nine positions are searched in a one-eighth-pixel refinement.

9. The system of claim 8 , wherein the half-pixel refinement comprises searching one integer-pixel search center pointed to by an integer motion vector and eight half-pixel positions in half pixel interpolation, and wherein the quarter-pixel refinement comprises searching one half-pixel position and eight quarter-pixel positions in quarter pixel interpolation, and wherein the one-eighth-pixel refinement comprises searching one quarter-pixel position and eight one-eighth-pixel positions in one-eighth pixel interpolation.

10. A system, comprising:

a processor configured to:

determine candidate integer level motion vectors for a video being encoded;

divide the candidate integer level motion vectors into a plurality of groups, wherein each group of the plurality of groups corresponds to a different single row of coding units of the frame of the video; and

process a group of the plurality of groups by one of a plurality of parallel row pipelines, wherein a parallel row pipeline is configured to process in parallel with other parallel row pipelines a different single row of coding units of the frame of the video, and wherein the plurality of parallel pipelines refine the candidate integer level motion vectors into candidate sub-pixel level motion vectors; and

a memory coupled to the processor and configured to provide the processor with instructions.

11. A method of encoding a video, comprising:

determining candidate integer level motion vectors for the video being encoded; and

dividing the candidate integer level motion vectors into a plurality of groups, wherein each group of the plurality of groups corresponds to a different single row of coding units of the frame of the video; and

processing a group of the plurality of groups by one of a plurality of parallel row pipelines, wherein a parallel row pipeline is configured to process in parallel with other parallel row pipelines a different single row of coding units of the frame of the video, and wherein the plurality of parallel pipelines refine the candidate integer level motion vectors into candidate sub-pixel level motion vectors.

12. The method of claim 11 , further comprising computing a fractional motion vector and sending the fractional motion vector to an inter prediction module of a mode decision module.

13. The method of claim 11 , wherein a coding unit of the frame of the video comprises a largest coding unit that may be divided into smaller prediction units.

14. The method of claim 11 , wherein in the event of data dependencies in processing the coding units across different rows of coding units of the frame of the video, processing a certain coding unit in a particular row and column after the processing of another coding unit in a previous row and a particular column is finished by a different parallel pipeline.

15. The method of claim 11 , wherein a parallel pipeline comprises parallel partition unit pipelines, and wherein a partition unit pipeline is configured to process in parallel with the other partition unit pipelines different partition units, and wherein the partition unit pipeline is configured to process a different set of partition units.

16. The method of claim 15 , wherein a first parallel partition unit pipeline is configured to process partition units with the smallest partition unit shapes, and wherein a second parallel partition unit pipeline is configured to process partition units with the remaining larger partition unit shapes.

17. The method of claim 15 , wherein a parallel partition unit pipeline comprises pipelined half pixel interpolation, quarter pixel interpolation, and one-eighth pixel interpolation for refining the candidate integer level motion vectors into the candidate sub-pixel level motion vectors.

18. The method of claim 17 , wherein nine positions are searched in a half-pixel refinement, and wherein nine positions are searched in a quarter-pixel refinement, and wherein nine positions are searched in a one-eighth-pixel refinement.

19. The method of claim 18 , wherein the half-pixel refinement comprises searching one integer-pixel search center pointed to by an integer motion vector and eight half-pixel positions in half pixel interpolation, and wherein the quarter-pixel refinement comprises searching one half-pixel position and eight quarter-pixel positions in quarter pixel interpolation, and wherein the one-eighth-pixel refinement comprises searching one quarter-pixel position and eight one-eighth-pixel positions in one-eighth pixel interpolation.

20. The system of claim 10 , wherein in the event of data dependencies in processing the coding units across different rows of coding units of the frame of the video, the processing of a certain coding unit in a particular row and column is performed after the processing of another coding unit in a previous row and a particular column is finished by a different parallel pipeline.

Assignments (2)
CHANGE OF NAME Recorded Nov 19, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058214/0351 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2021
From: SRIADIBHATLA, KAMESWARA KISHORE; CHEN, YUNQING; LAN, JUNQIANG; LEWIS, ADRIAN STAFFORD; SUNIL, ANIL MUTHIRAPARAMPIL
To: FACEBOOK, INC.
Reel/Frame 057740/0918 →