IP Library Granted Patent US 11,582,462
Granted Patent B1
US 11,582,462 · App. 17/375,408 · Granted Feb 14, 2023

Constraint-modified selection of video encoding configurations

Inventors: Jae Hoon Kim (Los Gatos, CA); Cosmin Vasile Stejerean (Las Vegas, NV); Shankar Lakshmi Regunathan (Redmond, WA); Pankaj Sethi (Palo Alto, CA)
Assignee: Meta Platforms, Inc.
H04N19/154H04N19/146
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,582,462
App. No.
17/375,408
Granted
Feb 14, 2023
Kind
B1
Abstract

A video to be encoded to a plurality of different target encodings for bandwidth adaptive serving is received. The video is encoded into a plurality of different candidate encodings using different candidate encoding parameters. A quality metric is determined for each of the plurality of different candidate encodings. One or more different target quality metrics are selected for a first portion of the different target encodings based at least in part on one or more specified constraints for one or more target devices. One or more different target quality metrics are selected for a second portion of the different target encodings based at least in part on the determined quality metrics of the different candidate encodings. Based at least in part on the selected different target quality metrics for the first portion and the second portion, the plurality of different target encodings of the video is generated.

Claims (49)

1. A method, comprising:

receiving a video to be encoded to a plurality of different target encodings for bandwidth adaptive serving;

encoding the video into a plurality of different candidate encodings using different candidate encoding parameters as part of performing a modified convex hull analysis to determine the plurality of different target encodings of the video, wherein performing the modified convex hull analysis includes:

determining a quality metric for each of the plurality of different candidate encodings;

determining a plurality of rate distortion curves based at least in part on the determined quality metrics;

for a first portion of the modified convex hull analysis corresponding to a first group of one or more bitrate lanes, determining a first convex hull using a subset of the determined rate distortion curves that removes one or more rate distortion curves from the determined rate distortion curves based at least in part on one or more specified constraints for one or more target devices;

selecting one or more different target quality metrics for a first portion of the different target encodings based on selecting points along the first determined convex hull;

for a second portion of the modified convex hull analysis corresponding to a second group of one or more bitrate lanes, determining a second convex hull using a combination of rate distortion curves different from the subset of the determined rate distortion curves used to determine the first convex hull; and

selecting one or more different target quality metrics for a second portion of the different target encodings based on selecting points along the second determined convex hull; and

based at least in part on the selected one or more different target quality metrics for the first portion of the different target encodings and the second portion of the different target encodings, generating the plurality of different target encodings of the video.

2. The method of claim 1 , wherein the different target encodings correspond to different hardware display resolutions for viewing the video.

3. The method of claim 1 , wherein the different candidate encoding parameters include video encoding resolution.

4. The method of claim 1 , wherein the different candidate encoding parameters include a specific encoding parameter that affects video encoding bitrate and video quality.

5. The method of claim 1 , wherein encoding the video into the plurality of different candidate encodings includes decompressing the video into a plurality of uncompressed images.

6. The method of claim 5 , wherein encoding the video into the plurality of different candidate encodings further includes resizing the decompressed video into a plurality of different video resolutions.

7. The method of claim 1 , wherein encoding the video into the plurality of different candidate encodings includes generating candidate encodings parameterized by video encoding resolutions and internal codec parameters.

8. The method of claim 1 , wherein the quality metric is based at least in part on a perceptual metric that reflects quality as perceived by humans.

9. The method of claim 1 , further comprising determining a bitrate corresponding to the quality metric for each of the plurality of different candidate encodings.

10. The method of claim 1 , wherein the one or more specified constraints includes a maximum allowed bitrate for a specified quality lane or bitrate lane.

11. The method of claim 1 , wherein selecting the one or more different target quality metrics for the first portion includes activating a rate control codec parameter.

12. The method of claim 1 , wherein the one or more specified constraints are associated with a requirement that a specified resolution be represented in the plurality of different target encodings.

13. The method of claim 1 , wherein selecting the one or more different target quality metrics for the first portion is associated with excluding specified rate distortion curves from a convex hull analysis.

14. The method of claim 1 , wherein selecting the one or more different target quality metrics for the first portion of the different target encodings includes determining pareto efficient points along the first determined convex hull.

15. The method of claim 1 , wherein selecting the one or more different target quality metrics for the second portion of the different target encodings includes determining pareto efficient points along the second determined convex hull.

16. The method of claim 1 , wherein selecting the one or more different target quality metrics for the second portion of the different target encodings includes determining a number of quality lanes to span across a target quality range.

17. The method of claim 1 , wherein selecting the one or more different target quality metrics for the second portion of the different target encodings includes determining whether spacings between bitrates corresponding to the target quality metrics for the second portion of the different target encodings exceed a specified threshold.

18. The method of claim 1 , further comprising transmitting a plurality of different video bitstreams corresponding to the plurality of different target encodings along a digital network.

19. A system, comprising:

one or more processors configured to:

receive a video to be encoded to a plurality of different target encodings for bandwidth adaptive serving;

encode the video into a plurality of different candidate encodings using different candidate encoding parameters as part of performing a modified convex hull analysis to determine the plurality of different target encodings of the video, wherein performing the modified convex hull analysis includes:

determining a quality metric for each of the plurality of different candidate encodings;

determining a plurality of rate distortion curves based at least in part on the determined quality metrics;

for a first portion of the modified convex hull analysis corresponding to a first group of one or more bitrate lanes, determining a first convex hull using a subset of the determined rate distortion curves that removes one or more rate distortion curves from the determined rate distortion curves based at least in part on one or more specified constraints for one or more target devices;

selecting one or more different target quality metrics for a first portion of the different target encodings based on selecting points along the first determined convex hull;

for a second portion of the modified convex hull analysis corresponding to a second group of one or more bitrate lanes, determining a second convex hull using a combination of rate distortion curves different from the subset of the determined rate distortion curves used to determine the first convex hull; and

selecting one or more different target quality metrics for a second portion of the different target encodings based on selecting points along the second determined convex hull; and

based at least in part on the selected one or more different target quality metrics for the first portion of the different target encodings and the second portion of the different target encodings, generate the plurality of different target encodings of the video; and

a memory coupled to at least one of the one or more processors and configured to provide at least one of the one or more processors with instructions.

20. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

receiving a video to be encoded to a plurality of different target encodings for bandwidth adaptive serving;

encoding the video into a plurality of different candidate encodings using different candidate encoding parameters as part of performing a modified convex hull analysis to determine the plurality of different target encodings of the video, wherein performing the modified convex hull analysis includes:

determining a quality metric for each of the plurality of different candidate encodings;

determining a plurality of rate distortion curves based at least in part on the determined quality metrics;

for a first portion of the modified convex hull analysis corresponding to a first group of one or more bitrate lanes, determining a first convex hull using a subset of the determined rate distortion curves that removes one or more rate distortion curves from the determined rate distortion curves based at least in part on one or more specified constraints for one or more target devices;

selecting one or more different target quality metrics for a first portion of the different target encodings based on selecting points along the first determined convex hull;

for a second portion of the modified convex hull analysis corresponding to a second group of one or more bitrate lanes, determining a second convex hull using a combination of rate distortion curves different from the subset of the determined rate distortion curves used to determine the first convex hull; and

selecting one or more different target quality metrics for a second portion of the different target encodings based on selecting points along the second determined convex hull; and

based at least in part on the selected one or more different target quality metrics for the first portion of the different target encodings and the second portion of the different target encodings, generating the plurality of different target encodings of the video.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2022
From: KIM, JAE HOON; STEJEREAN, COSMIN VASILE; REGUNATHAN, SHANKAR LAKSHMI; SETHI, PANKAJ
To: FACEBOOK, INC.
Reel/Frame 061691/0024 →
CHANGE OF NAME Recorded Nov 19, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058214/0351 →