IP Library › Granted Patent US 10,917,644
Granted Patent B2
US 10,917,644 · App. 15/902,976 · Granted Feb 9, 2021

Iterative techniques for encoding video content

Inventor: Ioannis Katsavounidis (San Jose, CA)
Assignee: NETFLIX, INC.
H04N19/147G11B20/00007G11B27/3081G11B27/34H04L65/4084H04L65/602H04L65/604H04L65/607H04L65/80H04N19/124H04N19/172H04N19/177H04N19/179H04N19/192H04N19/198H04N19/59H04N21/23418H04N21/23439H04N21/23805H04N21/234363H04N21/8456G11B2020/00072H04N19/126H04N19/15H04N19/40H04N21/8455
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,917,644
App. No.
15/902,976
Granted
Feb 9, 2021
Kind
B2
Abstract

In various embodiments, an iterative encoding application encodes a source video sequence. The encoding optimization application generates a set of shot encode points based on a set of encoding points and a first shot sequence included in the source video sequence. Each shot encode point is associated with a different encoded shot sequence. The encoding optimization application performs convex hull operation(s) across the set of shot encode points to generate a first convex hull associated with the first shot sequence. Subsequently, the encoding optimization application generates encoded video sequences based on the first convex hull and a second convex hull associated with a second shot sequence included in the source video sequence. The encoding optimization application computes a new encoding point based on the encoded video sequences and a target value for a first video metric and then generates an optimized encoded video sequence based on the new encoding point.

Claims (84)

1. A computer-implemented method, comprising:

generating a first set of shot encode points based on a first set of encoding points and a first shot sequence included in a source video sequence that is associated with a media title, wherein each shot encode point is associated with a different encoded shot sequence;

performing one or more operations across the first set of shot encode points to generate a first convex hull that is associated with the first shot sequence;

generating a plurality of encoded video sequences based on (i) the first convex hull and (ii) a second convex hull that is associated with a second shot sequence included in the source video sequence;

computing a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video sequences and a target value for a first video metric; and

generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently streamed to an endpoint device during a playback of the media title.

2. The computer-implemented method of claim 1 , wherein generating the optimized encoded video sequence comprises:

generating a first encoded shot sequence based on the first shot sequence and the first encoding point;

generating a second encoded shot sequence based on the second shot sequence and a second encoding point that is not equal to the first encoding point; and

aggregating the first encoded shot sequence and the second encoded shot sequence.

3. The computer-implemented method of claim 1 , wherein performing the one or more operations comprises:

determining a region that includes the first set of shot encode points;

identifying a boundary of the region, wherein no shot encode points included in the first set of shot encode points are located on a first side of the boundary; and

discarding any shot encode points included in the first set of shot encode points that are not located along the boundary to generate the first convex hull.

4. The computer-implemented method of claim 1 , wherein generating the plurality of encoded video sequences comprises:

computing a first slope value between a first shot encode point and a second shot encode point, wherein both the first shot encode point and the second shot encode point are included in the first convex hull;

selecting a first encoded shot sequence associated with the second shot encode point based on the first slope value; and

aggregating the first encoded shot sequence with a second encoded shot sequence to generate a first encoded video sequence, wherein the second encoded shot sequence is associated with a third shot encode point included in the second convex hull.

5. The computer-implemented method of claim 1 , wherein computing the first encoding point comprises:

generating a global convex hull based on the plurality of encoded video sequences, wherein the global convex hull includes a first set of video encode points;

selecting a first video encode point included in the first set of video encode points based on the target value; and

determining the first encoding point based on the first video encode point and the first convex hull.

6. The computer-implemented method of claim 1 , wherein computing the first encoding point comprises:

identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value;

identifying a first shot encode point that is associated with the first encoded video sequence and is included in the first convex hull;

identifying a second shot encode point included in the first convex hull based on a distance to the first shot encode point; and

performing at least one arithmetic operation based on the first shot encode point and the second shot encode point.

7. The computer-implemented method of claim 6 , wherein performing the at least one arithmetic operation comprises computing an average of a first value of an encoding parameter and a second value of the encoding parameter, and wherein the first value of the encoding parameter is associated with the first shot encode point and the second value of the encoding parameter is associated with the second shot encode point.

8. The computer-implemented method of claim 1 , wherein computing the first encoding point comprises:

identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value;

identifying a first shot encode point that is associated with the first encoded video sequence and is included in the first convex hull;

identifying a second shot encode point included in the first convex hull based on a distance between the first shot encode point and the second shot encode point;

determining that a first resolution associated with the first shot encode point is not equal to a second resolution associated with the second shot encode point;

setting, based on the first resolution and the second resolution, an endpoint value of an encoding parameter equal to either a maximum value of the encoding parameter or a minimum value of the encoding parameter; and

performing at least one arithmetic operation between a first value of the encoding parameter that is associated with the first shot encode point and the endpoint value of the encoding parameter.

9. The computer-implemented method of claim 1 , wherein the first video metric comprises a bitrate, a peak signal-to-noise-ratio (PSNR), a linear video multimethod assessment fusion (VMAF) metric, a harmonic VMAF (VMAFh) metric, or a distortion metric.

10. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform the steps of:

generating a first set of shot encode points based on a first set of encoding points and a first shot sequence included in a source video sequence that is associated with a media title, wherein each shot encode point is associated with a different encoded shot sequence;

performing one or more operations across the first set of shot encode points to generate a first convex hull that is associated with the first shot sequence;

generating a global convex hull based on (i) the first convex hull and (ii) a second convex hull that is associated with a second shot sequence included in the source video sequence;

computing a first encoding point that is not included in the first set of encoding points based on a target value for a first video metric, the global convex hull, and the first convex hull; and

generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently streamed to an endpoint device during a playback of the media title.

11. The computer-readable storage medium of claim 10 , wherein generating the optimized encoded video sequence comprises:

generating a first encoded shot sequence based on the first shot sequence and the first encoding point;

generating a second encoded shot sequence based on the second shot sequence and a second encoding point that is not equal to the first encoding point; and

aggregating the first encoded shot sequence and the second encoded shot sequence.

12. The computer-readable storage medium of claim 10 , wherein performing the one or more operations comprises:

determining a region that includes the first set of shot encode points;

identifying a boundary of the region, wherein no shot encode points included in the first set of shot encode points are located on a first side of the boundary; and

discarding any shot encode points included in the first set of shot encode points that are not located along the boundary to generate the first convex hull.

13. The computer-readable storage medium of claim 12 , wherein each shot encode point included in the first convex hull is associated with both a bitrate and a value for either a distortion metric or a visual quality metric.

14. The computer-readable storage medium of claim 10 , wherein generating the global convex hull comprises:

computing a first slope value between a first shot encode point and a second shot encode point, wherein both the first shot encode point and the second shot encode point are included in the first convex hull;

selecting a first encoded shot sequence associated with the second shot encode point based on the first slope value;

aggregating the first encoded shot sequence with a second encoded shot sequence to generate a first encoded video sequence, wherein the second encoded shot sequence is associated with a third shot encode point included in the second convex hull;

generating a first video encode point based on the first encoded video sequence; and

adding the first video encode point to a partial global convex hull to generate the global convex hull.

15. The computer-readable storage medium of claim 10 , wherein computing the first encoding point comprises:

selecting a first video encode point included in the global convex hull based on the target value; and

selecting a first shot encode point included in the first convex hull based on the first video encode point;

identifying a second shot encode point included in the first convex hull based on a distance between the first shot encode point and the second shot encode point; and

performing at least one arithmetic operation based on the first shot encode point and the second shot encode point.

16. The computer-readable storage medium of claim 15 , wherein performing the at least one arithmetic operation comprises computing an average of a first resolution associated with the first shot encode point and a second resolution associated with the second shot encode point.

17. The computer-readable storage medium of claim 10 , wherein computing the first encoding point comprises:

selecting a first video encode point included in the global convex hull based on the target value; and

selecting a first shot encode point included in the first convex hull based on the first video encode point;

identifying a second shot encode point included in the first convex hull based on a distance between the first shot encode point and the second shot encode point;

determining that a first resolution associated with the first shot encode point is not equal to a second resolution associated with the second shot encode point;

setting, based on the first resolution and the second resolution, an endpoint value of an encoding parameter equal to either a maximum value of the encoding parameter or a minimum value of the encoding parameter; and

performing at least one arithmetic operation between a first value of the encoding parameter that is associated with the first shot encode point and the endpoint value of the encoding parameter.

18. The computer-readable storage medium of claim 10 , wherein the first encoding point specifies at least one of a resolution and an encoding parameter.

19. A system, comprising:

a memory storing instructions; and

a processor that is coupled to the memory and, when executing the instructions, is configured to:

partition a video sequence that is associated with a media title into a plurality of frame sets,

generate a first set of shot encode points based on a first set of encoding points and a first frame set included in the plurality of frame sets, wherein each shot encode point is associated with a different encoded frame set,

perform one or more operations across the first set of shot encode points to generate a first convex hull that is associated with the first frame set,

generate a plurality of encoded video sequences based on (i) the first convex hull and (ii) a second convex hull that is associated with a second frame set included in the plurality of frame sets,

compute a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video sequences and a target value for a first video metric, and

generate an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently streamed to an endpoint device during a playback of the media title.

20. The system of claim 19 , wherein the processor is configured to generate the optimized encoded video sequence by:

generating a first encoded frame set based on the first frame set and the first encoding point;

generating a second encoded frame set based on the second frame set and a second encoding point that is not equal to the first encoding point; and

aggregating the first encoded frame set and the second encoded frame set.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2018
From: KATSAVOUNIDIS, IOANNIS
To: NETFLIX, INC.
Reel/Frame 045093/0444 →
Continuity (4)
Provisional Application 62462873 · Feb 23, 2017
Provisional Application 62534170 · Jul 18, 2017
Provisional Application 62550517 · Aug 25, 2017
Related Publication 20180242002A1 · Aug 23, 2018