IP Library › Granted Patent US 10,911,791
Granted Patent B2
US 10,911,791 · App. 16/244,033 · Granted Feb 2, 2021

Optimizing encoding operations when generating a buffer-constrained version of a media title

Inventor: Ioannis Katsavounidis (San Jose, CA)
Assignee: NETFLIX, INC.
H04N21/231H04N21/235H04N21/236H04N21/2662
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,911,791
App. No.
16/244,033
Granted
Feb 2, 2021
Kind
B2
Abstract

In various embodiments, a buffer-based encoding application generates a first convex hull of subsequence encode points based on multiple encoding points and a first subsequence associated with a media title. The buffer-based encoding application then generates a first global convex hull of media encode points based on a transmission buffer constraint, the first convex hull, and a second global convex hull of media encode points. Notably, the second global convex hull is associated with a portion of the media title that occurs before the first subsequence in a playback order for the media title. Subsequently, the subsequence-based encoding application selects a first media encode point included in the first global convex hull based on a media metric and determines a first encoded media sequence based on the selected media encode point. The first encoded media sequence is subsequently streamed to an endpoint device during playback of the media title.

Claims (59)

1. A computer-implemented method, comprising:

generating a first convex hull of subsequence encode points based on a first set of encoding points and a first subsequence associated with a media title;

generating a first global convex hull of media encode points based on a transmission buffer constraint, the first convex hull, and a second global convex hull of media encode points associated with a portion of the media title that occurs before the first subsequence in a playback order for the media title;

selecting a first media encode point included in the first global convex hull based on a media metric; and

determining a first encoded media sequence based on the first media encode point, wherein the first encoded media sequence is subsequently streamed to an endpoint device during playback of the media title.

2. The computer-implemented method of claim 1 , wherein the transmission buffer constraint comprises a constant bitrate video buffer verifier (VBV) constraint or a variable bitrate VBV constraint.

3. The computer-implemented method of claim 1 , further comprising computing a pre-buffering time based on a minimum buffer occupancy associated with the first encoded media sequence and a buffer size associated with the transmission buffer constraint.

4. The computer-implemented method of claim 1 , wherein generating the first global convex hull comprises:

for each combination of a media encode point included in the second global convex hull and a subsequence encode point included in the first convex hull, generating a different media encode point included in a set of media encode points;

performing one or more filtering operations on the set of media encode points based on the transmission buffer constraint to generate a set of compliant media encode points; and

performing one or more convex hull operations across the set of compliant media encode points to generate the first global convex hull.

5. The computer-implemented method of claim 1 , wherein the media metric comprises a bitrate, and selecting the first media encode point comprises determining that a first average bitrate associated with the first media encode point is greater than average bitrates associated with all other media encode points included in the first global convex hull.

6. The computer-implemented method of claim 1 , wherein generating the first global convex hull comprises:

generating a second encoded media sequence based on a second media encode point included in the second global convex hull and a first subsequence encode point included in the first convex hull;

computing a minimum buffer occupancy and a maximum buffer occupancy based on the second encoded media sequence and a transmission rate associated with the transmission buffer constraint;

determining that a difference between the maximum buffer occupancy and the minimum buffer occupancy does not exceed a buffer size associated with the transmission buffer constraint; and

in response to determining that the difference does not exceed the buffer size, adding a second media encode point associated with the second encoded media sequence to the first global convex hull.

7. The computer-implemented method of claim 1 , wherein generating the first convex hull comprises:

generating a first set of subsequence encode points based on the first set of encoding points and the first subsequence;

determining a region that includes the first set of subsequence encode points;

identifying a boundary associated with the region, wherein no subsequence encode points included in the first set of subsequence encode points are located on a first side of the boundary; and

discarding all subsequence encode points included in the first set of subsequence encode points that are not located along the boundary.

8. The computer-implemented method of claim 1 , wherein each encoding point included in the first set of encoding points specifies a different combination of a resolution and a rate control value.

9. The computer-implemented method of claim 1 , further comprising partitioning a source media sequence associated with the media title into a plurality of subsequences based on a plurality of shot changes, wherein the plurality of subsequences includes the first subsequence.

10. The computer-implemented method of claim 1 , wherein the media title comprises at least one of video content or audio content.

11. One or more non-transitory computer readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

encoding a first subsequence associated with a media title across a first set of encoding points to generate a first set of subsequence encode points, wherein each subsequence encode point is associated with a different encoded subsequence;

performing one or more convex hull operations across the first set of subsequence encode points to compute a first convex hull of subsequence encode points based on at least two media metrics;

generating a first global convex hull of media encode points based on a transmission buffer constraint, the first convex hull, and a second global convex hull of media encode points associated with one or more subsequences associated with the media title that occur before the first subsequence in a playback order for the media title;

selecting a first media encode point included in the first global convex hull based on a first media metric; and

determining a first encoded media sequence based on the first media encode point, wherein the first encoded media sequence is subsequently streamed to an endpoint device during playback of the media title.

12. The one or more non-transitory computer readable media of claim 11 , wherein the transmission buffer constraint specifies at least one of a transmission rate or a buffer size.

13. The one or more non-transitory computer readable media of claim 11 , the steps further comprising computing a pre-buffering time based on a minimum buffer occupancy associated with the first encoded media sequence and a buffer size associated with the transmission buffer constraint.

14. The one or more non-transitory computer readable media of claim 11 , wherein generating the first global convex hull comprises:

generating a second media encode point based on a third media encode point included in the second global convex hull and a first subsequence encode point included in the first convex hull;

generating a fourth media encode point based on the third media encode point and a second subsequence encode point included in the first convex hull;

adding the second media encode point but not the fourth media encode point to a set of compliant encode points based on the transmission buffer constraint; and

performing one or more convex hull operations across the set of compliant media encode points to generate the first global convex hull.

15. The one or more non-transitory computer readable media of claim 11 , wherein the first media metric comprises a bitrate, and selecting the first media encode point comprises determining that a first average bitrate associated with the first media encode point is greater than average bitrates associated with all other media encode points included in the first global convex hull.

16. The one or more non-transitory computer readable media of claim 11 , wherein generating the first global convex hull comprises:

generating a second encoded media sequence based on a second media encode point included in the second global convex hull and a first subsequence encode point included in the first convex hull;

computing a minimum buffer occupancy and a maximum buffer occupancy based on the second encoded media sequence and a transmission rate associated with the transmission buffer constraint;

determining that a difference between the maximum buffer occupancy and the minimum buffer occupancy does not exceed a buffer size associated with the transmission buffer constraint; and

in response to determining that the difference does not exceed the buffer size, adding a second media encode point associated with the second encoded media sequence to the first global convex hull.

17. The one or more non-transitory computer readable media of claim 11 , wherein generating the first global convex hull comprises:

generating a set of compliant media encode points based on the transmission buffer constraint, the first convex hull, and the second global convex hull;

determining a region that includes the set of compliant media encode points;

identifying a boundary associated with the region, wherein no compliant media encode points included in the set of compliant media encode points are located on a first side of the boundary; and

discarding all compliant media encode points included in the set of compliant media encode points that are not located along the boundary.

18. The one or more non-transitory computer readable media of claim 11 , wherein the first encoded media sequence includes a first encoded subsequence and a second encoded subsequence, and wherein the first encoded subsequence is associated with a first encoding point included in the first set of encoding points and the second encoded subsequence is associated with a second encoding point included in the first set of encoding points that is not equal to the first encoding point.

19. The one or more non-transitory computer readable media of claim 11 , wherein the first subsequence associated with the media title comprises a shot sequence.

20. A system, comprising:

one or more memories storing instructions; and

one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to:

partition a source media sequence associated with a media title into a plurality of subsequences based on a plurality of shot changes;

generate a first convex hull of subsequence encode points based on a first set of encoding points and a first subsequence included in the plurality of subsequences, wherein each subsequence encode point is associated with a different encoded subsequence;

generate a first global convex hull of media encode points based on a transmission buffer constraint, the first convex hull, and a second global convex hull of media encode points associated with a portion of the media title that occurs before the first subsequence in a playback order for the media title;

select a first media encode point included in the first global convex hull based on a media metric; and

determine a first encoded media sequence based on the first media encode point, wherein the first encoded media sequence is subsequently streamed to an endpoint device during playback of the media title.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2019
From: KATSAVOUNIDIS, IOANNIS
To: NETFLIX, INC.
Reel/Frame 048481/0776 →
Continuity (1)
Related Publication 20200221141A1 · Jul 9, 2020
Cited By (1)
US 12,464,147