IP Library › Granted Patent US 11,924,435
Granted Patent B2
US 11,924,435 · App. 16/875,756 · Granted Mar 5, 2024

High quality advanced neighbor management encoder architecture

Inventors: Srinivasan Embar Raghukrishnan (Folsom, CA); Jason Tanner (Folsom, CA); Naiqian Lu (Fremont, OR)
Assignee: Intel Corporation
H04N19/139H04N19/103H04N19/119H04N19/184H04N19/436
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,924,435
App. No.
16/875,756
Granted
Mar 5, 2024
Kind
B2
Abstract

Techniques related to parallel partitioning and coding mode selection for improved video coding throughput are discussed. Such techniques include performing parallel partitioning and coding mode selection for a lower-right coding unit of a first largest coding unit and an upper-left coding unit of a second largest coding unit to the right of the first largest coding unit and, immediately subsequent thereto, performing parallel partitioning and coding mode selection for a lower-left coding unit and an upper-right coding unit of the second largest coding unit.

Claims (51)

1. A device for video coding comprising:

a memory to store at least a portion of a video frame, the video frame comprising a first largest coding unit (LCU) and a second LCU horizontally aligned with and immediately to the right of the first LCU, wherein each of the first and second LCUs comprises upper-left, upper-right, lower-left, and lower-right coding units (CUs); and

one or more processors coupled to the memory, the one or more processors to:

perform first integer motion estimation (IME) for the upper-left CU of the second LCU and second IME for the lower-right CU of the first LCU in parallel;

perform, immediately subsequent to the first and second IMEs, third IME for the upper-right CU of the second LCU and fourth IME for the lower-left CU of the second LCU in parallel, wherein the first IME for the upper-left CU of the second LCU uses a first resultant motion vector corresponding to the upper-right CU of the first LCU as a motion vector merge candidate and disables usage of a motion vector merge candidate corresponding to the first LCU;

perform first partitioning and coding mode selections for the upper-left CU of the second LCU and second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel;

perform, immediately subsequent to the first and second partitioning and coding mode selections, third partitioning and coding mode selections for the upper-right CU of the second LCU and fourth partitioning and coding mode selections for the lower-left CU of the second LCU in parallel, wherein the first, second, third, and fourth partitioning and coding mode selections are based on the first, second, third, and fourth IMEs, respectively; and

encode the first and second LCUs into a bitstream using the first, second, third, and fourth partitioning and coding mode selections.

2. The device of claim 1 , wherein the one or more processors to perform the fourth IME for the lower-left CU of the second LCU disables usage of a second motion vector merge candidate corresponding to the upper-right CU of the second LCU.

3. The device of claim 1 , the one or more processors to:

determine whether the first partitioning and coding mode selections for the upper-left CU of the second LCU comprises no partitioning and the third partitioning and coding mode selections for the upper-right CU of the second LCU comprises no partitioning; and

perform a merge motion vector evaluation for the second LCU in response to the first and third partitioning and coding mode selections comprising no partitionings, or

bypass the merge motion vector evaluation for the second LCU in response to either or both of the first and third partitioning and coding mode selections comprising additional partitionings.

4. The device of claim 3 , wherein the video frame comprises a third LCU horizontally aligned with and immediately to the right of the second LCU and wherein the third LCU comprises upper-left, upper-right, lower-left, and lower-right CUs, the one or more processors to:

perform, immediately subsequent to the third and fourth partitioning and coding mode selections, fifth partitioning and coding mode selections for the upper-left CU of the third LCU and sixth partitioning and coding mode selections for the lower-right CU of the second LCU in parallel, wherein the one or more processors to perform or bypass the merge motion vector evaluation for the second largest CU is in parallel with the fifth and sixth partitioning and coding mode selections.

5. The device of claim 1 , wherein the one or more processors to perform the first partitioning and coding mode selections for the upper-left CU of the second LCU and the second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel comprises breaking a bottom-left coding dependency from the upper-left CU of the second LCU to the lower-right CU of the LCU.

6. The device of claim 1 , wherein the one or more processors to perform the third partitioning and coding mode selections for the upper-right CU of the second LCU and the fourth partitioning and coding mode selections for the lower-left CU of the second LCU in parallel comprises breaking an upper-right coding dependency from the lower-left CU of the second LCU to the upper-right CU of the second LCU.

7. The device of claim 1 , wherein the first CU is in a first LCU wavefront and the second CU is in a second LCU wavefront.

8. The device of claim 1 , wherein the first partitioning and coding mode selections for the upper-left CU of the second LCU are performed by a first execution unit of the one or more processors and the second partitioning and coding mode selections for the lower-right CU of the first LCU are performed by a second execution unit of the one or more processors.

9. The device of claim 1 , wherein the first and second LCUs are 64×64 pixels and each of the upper-left, upper-right, lower-left, and lower-right coding units CUs are 32×32 pixels.

10. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to code video by:

receiving first and second largest coding units (LCUs) of a video frame, wherein the second LCU is horizontally aligned with and immediately to the right of the LCU and each of the first and second LCUs comprises upper-left, upper-right, lower-left, and lower-right coding units (CUs);

performing first integer motion estimation (IME) for the upper-left CU of the second LCU and second IME for the lower-right CU of the first LCU in parallel;

performing, immediately subsequent to the first and second IMEs, third IME for the upper-right CU of the second LCU and fourth IME for the lower-left CU of the second LCU in parallel, wherein the first IME for the upper-left CU of the second LCU uses a first resultant motion vector corresponding to the upper-right CU of the first LCU as a motion vector merge candidate and disables usage of a motion vector merge candidate corresponding to the first LCU;

performing first partitioning and coding mode selections for the upper-left CU of the second LCU and second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel;

performing, immediately subsequent to the first and second partitioning and coding mode selections, third partitioning and coding mode selections for the upper-right CU of the second LCU and fourth partitioning and coding mode selections for the lower-left CU of the second LCU in parallel, wherein the first, second, third, and fourth partitioning and coding mode selections are based on the first, second, third, and fourth IMEs, respectively; and

encoding the first and second LCUs into a bitstream using the first, second, third, and fourth partitioning and coding mode selections.

11. The non-transitory machine readable medium of claim 10 , wherein performing the fourth IME for the lower-left CU of the second LCU disables usage of a second motion vector merge candidate corresponding to the upper-right CU of the second LCU.

12. The non-transitory machine readable medium of claim 10 , further comprising instructions that, in response to being executed on the computing device, cause the computing device to code video by:

determining whether the first partitioning and coding mode selections for the upper-left CU of the second LCU comprises no partitioning and the third partitioning and coding mode selections for the upper-right CU of the second LCU comprises no partitioning; and

performing a merge motion vector evaluation for the second LCU in response to the first and third partitioning and coding mode selections comprising no partitionings, or

bypassing the merge motion vector evaluation for the second LCU in response to either or both of the first and third partitioning and coding mode selections comprising additional partitionings.

13. The non-transitory machine readable medium of claim 10 , wherein performing the first partitioning and coding mode selections for the upper-left CU of the second LCU and the second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel comprises breaking a bottom-left coding dependency from the upper-left CU of the second LCU to the lower-right CU of the LCU or performing the third partitioning and coding mode selections for the upper-right CU of the second LCU and the fourth partitioning and coding mode selections for the lower-left CU of the second LCU in parallel comprises breaking an upper-right coding dependency from the lower-left CU of the second LCU to the upper-right CU of the second LCU.

14. A device for video coding, comprising:

a memory to store at least a portion of a video frame, the video frame comprising a first largest coding unit (LCU) and a second LCU horizontally aligned with and immediately to the right of the first LCU, wherein each of the first and second LCUs comprises upper-left, upper-right, lower-left, and lower-right coding units (CUs); and

one or more processors coupled to the memory, the one or more processors to:

perform first integer motion estimation (IME) for the upper-left CU of the second LCU and second IME for the lower-right CU of the first LCU in parallel;

perform, immediately subsequent to the first and second IMEs, third IME for the upper-right CU of the second LCU and fourth IME for the lower-left CU of the second LCU in parallel, wherein the first IME for the upper-left CU of the second LCU uses a first resultant motion vector corresponding to the upper-right CU of the first LCU as a motion vector merge candidate and disables usage of a motion vector merge candidate corresponding to the first LCU; and

encode the first and second LCUs into a bitstream based on the first, second, third, and fourth IMEs.

15. The device of claim 14 , wherein the one or more processors to perform the fourth IME for the lower-left CU of the second LCU disables usage of a second motion vector merge candidate corresponding to the upper-right CU of the second LCU.

16. The device of claim 14 , the processor further to:

perform first partitioning and coding mode selections for the upper-left CU of the second LCU and second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel; and

perform third partitioning and coding mode selections for the upper-right CU of the second LCU and fourth partitioning and coding mode selections for the lower-left CU of the LCU in parallel.

17. The device of claim 16 , the one or more processors to:

determine whether the first partitioning and coding mode selections for the upper-left CU of the second LCU comprises no partitioning and the third partitioning and coding mode selections for the upper-right CU of the second LCU comprises no partitioning; and

perform a merge motion vector evaluation for the second LCU in response to the first and third partitioning and coding mode selections comprising no partitionings, or

bypass the merge motion vector evaluation for the second LCU in response to either or both of the first and third partitioning and coding mode selections comprising additional partitionings.

18. The device of claim 17 , wherein the video frame comprises a third LCU horizontally aligned with and immediately to the right of the second LCU and wherein the third LCU comprises upper-left, upper-right, lower-left, and lower-right CUs, the one or more processors to:

perform, immediately subsequent to the third and fourth partitioning and coding mode selections, fifth partitioning and coding mode selections for the upper-left CU of the third LCU and sixth partitioning and coding mode selections for the lower-right CU of the second LCU in parallel, wherein the one or more processors to perform or bypass the merge motion vector evaluation for the second largest CU is in parallel with the fifth and sixth partitioning and coding mode selections.

19. The device of claim 16 , wherein the one or more processors to perform the first partitioning and coding mode selections for the upper-left CU of the second LCU and the second partitioning and coding mode selections for the lower-right CU of the first LCU in parallel comprises breaking a bottom-left coding dependency from the upper-left CU of the second LCU to the lower-right CU of the LCU.

20. The device of claim 16 , wherein the one or more processors to perform the third partitioning and coding mode selections for the upper-right CU of the second LCU and the fourth partitioning and coding mode selections for the lower-left CU of the second LCU in parallel comprises breaking an upper-right coding dependency from the lower-left CU of the second LCU to the upper-right CU of the second LCU.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2020
From: EMBAR RAGHUKRISHNAN, SRINIVASAN; TANNER, JASON; LU, NAIQIAN
To: INTEL CORPORATION
Reel/Frame 052676/0859 →
Continuity (1)
Related Publication 20200280722A1 · Sep 3, 2020