IP Library Granted Patent US 11,425,393
Granted Patent B1
US 11,425,393 · App. 17/344,170 · Granted Aug 23, 2022

Hardware optimization of rate calculation in rate distortion optimization for video encoding

Inventors: Zhao Wang (Newark, CA); Srikanth Alaparthi (Fremont, CA); Yunqing Chen (Los Altos, CA); Baheerathan Anandharengan (Milpitas, CA); Gaurang Chaudhari (Sunnyvale, CA); Junqiang Lan (Fremont, CA); Harikrishna Madadi Reddy (San Jose, CA); Prahlad Rao Venkatapuram (Saratoga, CA)
Assignee: Meta Platforms, Inc.
H04N19/149H04N19/103H04N19/147H04N19/159
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,425,393
App. No.
17/344,170
Granted
Aug 23, 2022
Kind
B1
Abstract

A system for calculating token rates for video encoding includes a plurality of different probability lookup tables implemented in hardware, wherein each of the probability lookup tables specifically corresponds to a different prediction mode of a video codec. The system includes an application-specific integrated circuit compute unit. For each candidate prediction mode among the different prediction modes, the application-specific integrated circuit is configured to determine a rate distortion cost (RD Cost) for a video. The application-specific integrated circuit is configured to select one of the plurality of different probability lookup tables that corresponds to the candidate prediction mode and use the selected one of the plurality of different probability lookup tables to calculate a corresponding token rate for the candidate prediction mode. The application-specific integrated circuit is configured to encode the video using a selected one of the different prediction modes determined based on the determined rate distortion costs.

Claims (39)

1. A method, comprising:

for each candidate prediction mode among different prediction modes of a video codec:

selecting by an application-specific integrated circuit one of a plurality of different probability lookup tables that corresponds to the candidate prediction mode;

using by the application-specific integrated circuit the selected one of the plurality of different probability lookup tables to calculate a corresponding token rate for the candidate prediction mode for a video to be encoded; and

using by the application-specific integrated circuit the corresponding token rate for the candidate prediction mode to determine a rate distortion cost for the video; and

encoding the video using a selected one of the different prediction modes determined based on the determined rate distortion costs.

2. The method of claim 1 , wherein a prediction mode comprises one or more parameters defined by a standardized video coding format, wherein the one or more parameters comprise a transform unit size, and wherein the standardized video coding format is selected from a group consisting of: H.262, H.264, H.265, Theora, RealVideo RV40, VP9, and AV1.

3. The method of claim 1 , wherein each of the plurality of different probability lookup tables comprises a partition of an original probability lookup table that is partitioned based on one or more parameters defined by a video coding standard.

4. The method of claim 3 , wherein the partitions of the original probability lookup table are partitioned based on a type of prediction that exploits a type of redundancy.

5. The method of claim 4 , wherein the type of redundancy comprises spatial redundancy.

6. The method of claim 4 , wherein the type of prediction comprises intra prediction.

7. The method of claim 4 , wherein the type of redundancy comprises temporal redundancy.

8. The method of claim 4 , wherein the type of prediction comprises inter prediction.

9. The method of claim 3 , wherein the partitions of the original probability lookup table are partitioned based on one or more transform unit sizes defined by the video coding standard.

10. The method of claim 9 , wherein the one or more transform unit sizes comprise 4×4 and 8×8.

11. A system, comprising:

a plurality of different probability lookup tables implemented in hardware, wherein each of the plurality of different probability lookup tables specifically corresponds to a different prediction mode of a video codec; and

an application-specific integrated circuit compute unit configured to:

for each candidate prediction mode among the different prediction modes of the video codec:

select one of the plurality of different probability lookup tables that corresponds to the candidate prediction mode;

use the selected one of the plurality of different probability lookup tables to calculate a corresponding token rate for the candidate prediction mode for a video to be encoded; and

use the corresponding token rate for the candidate prediction mode to determine a rate distortion cost for the video; and

encode the video using a selected one of the different prediction modes determined based on the determined rate distortion costs.

12. The system of claim 11 , wherein a prediction mode comprises one or more parameters defined by a standardized video coding format, wherein the one or more parameters comprise a transform unit size, and wherein the standardized video coding format is selected from a group consisting of: H.262, H.264, H.265, Theora, RealVideo RV40, VP9, and AV1.

13. The system of claim 11 , wherein each of the plurality of different probability lookup tables comprises a partition of an original probability lookup table that is partitioned based on one or more parameters defined by a standardized video coding format, and wherein the standardized video coding format is selected from a group consisting of: H.262, H.264, H.265, Theora, RealVideo RV40, VP9, and AV1.

14. The system of claim 13 , wherein the partitions of the original probability lookup table are partitioned based on a type of prediction that exploits a type of redundancy.

15. The system of claim 14 , wherein the type of redundancy comprises spatial redundancy.

16. The system of claim 14 , wherein the type of prediction comprises intra prediction.

17. The system of claim 14 , wherein the type of redundancy comprises temporal redundancy.

18. The system of claim 14 , wherein the type of prediction comprises inter prediction.

19. The system of claim 13 , wherein the partitions of the original probability lookup table are partitioned based on one or more transform unit sizes defined by the video coding standard.

20. A system, comprising:

A processor comprising logic to:

for each candidate prediction mode among different prediction modes of a video codec:

select one of a plurality of different probability lookup tables that corresponds to the candidate prediction mode;

use the selected one of the plurality of different probability lookup tables to calculate a corresponding token rate for the candidate prediction mode for a video to be encoded; and

use the corresponding token rate for the candidate prediction mode to determine a rate distortion cost for the video; and

encode the video using a selected one of the different prediction modes determined based on the determined rate distortion costs; and

a memory coupled to the processor and configured to provide the processor with instructions.

Assignments (2)
CHANGE OF NAME Recorded Nov 19, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058214/0351 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2021
From: WANG, ZHAO; ALAPARTHI, SRIKANTH; CHEN, YUNQING; ANANDHARENGAN, BAHEERATHAN; CHAUDHARI, GAURANG; LAN, JUNQIANG; REDDY, HARIKRISHNA MADADI; VENKATAPURAM, PRAHLAD RAO
To: FACEBOOK, INC.
Reel/Frame 057262/0801 →
Cited By (1)
US 12,615,383