IP Library Granted Patent US 11,910,042
Granted Patent B2
US 11,910,042 · App. 17/834,842 · Granted Feb 20, 2024

Dynamic allocation of compute capacity in video stream processing

Inventor: Neven Haltmayer (Sunnyvale, CA)
Assignee: Harmonic, Inc.
H04N21/2385H04N7/01H04N7/0806H04N19/127H04N19/154H04N19/172H04N19/179H04N21/23418H04N21/241H04N21/2405H04N21/6143H04N19/136H04N19/436
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,910,042
App. No.
17/834,842
Granted
Feb 20, 2024
Kind
B2
Abstract

Approaches for dynamically allocating compute capacity for processing a video stream. Video complexity information for two or more digital video streams actively being processed by one or more video encoders is determined at periodic intervals. Video complexity information describes the complexity of digital video carried by the digital video streams across a bounded number of consecutive digital frames which includes digital frames not yet processed by the one or more video encoders. A determination is made as to whether the compute capacity allocated for processing a particular digital video stream should be adjusted in some manner based on the determined video complexity information. The amount of compute capacity allocated for processing the particular digital video stream may be dynamically adjusted in response to maximizing a measure of optimal video quality calculated for the two or more digital video streams using, at least in part, the determined video complexity information.

Claims (48)

1. A non-transitory computer-readable storage medium storing one or more sequences of instructions for dynamically allocating compute capacity for use in encoding a video stream, which when executed by one or more processors, causes:

upon the occurrence of each periodic interval in a series of periodic intervals, performing: (a) determining, for a particular future periodic interval of said series of periodic intervals, video complexity information for two or more digital video streams actively being encoded at the same time, (b) determining whether an amount of the compute capacity allocated for encoding a particular digital video stream of the two or more digital video streams should be adjusted during said particular future periodic interval of said series of periodic intervals based on (i) the determined video complexity information and (ii) an assessment of impact to quality of said particular digital video stream resulting from making said adjustment to said amount of compute capacity, and (c) upon determining that the amount of compute capacity currently allocated for encoding said particular digital video stream of the two or more digital video streams should be adjusted during said future periodic interval, adjusting the amount of compute capacity currently allocated during said future periodic interval to encoding the particular digital video stream of the two or more digital video streams.

2. The non-transitory computer-readable storage medium of claim 1 , wherein each periodic interval in the series of periodic intervals corresponds to a frame of digital video in the particular video stream.

3. The non-transitory computer-readable storage medium of claim 1 , wherein each periodic interval in the series of periodic intervals corresponds to a period of time or a certain number of CPU clock cycles.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the determined video complexity information describes the complexity of digital video carried by the two or more digital video streams across a bounded number of consecutive digital frames, wherein said bounded number of consecutive digital frames includes digital frames not yet encoded.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the execution of the one or more sequences of instructions further causes:

dynamically adjusting a portion of the compute capacity allocated for encoding the particular digital video stream by maximizing a measure of optimal video quality calculated for the two or more digital video streams using, at least in part, the determined video complexity information.

6. The non-transitory computer-readable storage medium of claim 5 , wherein

maximizing the measure of optimal video quality for the two or more digital video streams comprises:

identifying one or more features in at least one of the two or more digital video streams in which support for will be reduced or ceased during said future periodic interval.

7. The non-transitory computer-readable storage medium of claim 5 , wherein

the measure of optimal video quality for the two or more digital video streams is determined by assessing the video quality of at least one of the two or more digital video streams at a greater importance than another digital video stream in the two or more digital video streams.

8. The non-transitory computer-readable storage medium of claim 1 , wherein the execution of the one or more sequences of instructions further causes:

dynamically increasing some portion of the compute capacity allocated to encode the particular digital video stream based on, at least in part, the determined video complexity information, wherein dynamically increasing comprising:

assessing a first impact of decreasing a first portion of the compute capacity allocated to encode at least one digital video stream of the two or more digital video streams;

assessing a second impact of increasing a second portion of the compute capacity to encode the particular digital video stream proportional to the first portion.

9. The non-transitory computer-readable storage medium of claim 1 , wherein at least two of the two or more video streams are not encoded on the same physical machine.

10. An apparatus for dynamically allocating compute capacity

for use in encoding a video stream, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums storing one or more sequences of instructions, which when executed, cause:

upon the occurrence of each periodic interval in a series of periodic intervals, performing: (a) determining, for a particular future periodic interval of said series of periodic intervals, video complexity information for two or more digital video streams actively being encoded at the same time, (b) determining whether an amount of the compute capacity allocated for encoding a particular digital video stream of the two or more digital video streams should be adjusted during said particular future periodic interval of said series of periodic intervals based on Ithe determined video complexity information and (ii) an assessment of impact to quality of said particular digital video stream resulting from making said adjustment to said amount of compute capacity, and (c) upon determining that the amount of compute capacity currently allocated for encoding said particular digital video stream of the two or more digital video streams should be adjusted during said future periodic interval, adjusting the amount of compute capacity currently allocated during said future periodic interval to encoding the particular digital video stream of the two or more digital video streams.

11. The apparatus of claim 10 , wherein each periodic interval in the series of periodic intervals corresponds to a frame of digital video in the particular video stream.

12. The apparatus of claim 10 , wherein each periodic interval in the series of periodic intervals corresponds to a period of time or a certain number of CPU clock cycles.

13. The apparatus of claim 10 , wherein the determined video complexity information describes the complexity of digital video carried by the two or more digital video streams across a bounded number of consecutive digital frames, wherein said bounded number of consecutive digital frames includes digital frames not yet encoded by said one or more video encoders.

14. The apparatus of claim 10 , wherein the execution of the one or more sequences of instructions further causes:

dynamically adjusting a portion of the compute capacity allocated for encoding the particular digital video stream by maximizing a measure of optimal video quality calculated for the two or more digital video streams using, at least in part, the determined video complexity information.

15. The apparatus of claim 14 , wherein maximizing the measure of optimal video quality for the two or more digital video streams comprises:

identifying one or more features in at least one of the two or more digital video streams in which support for will be reduced or ceased during said periodic interval.

16. The apparatus of claim 14 , wherein

the measure of optimal video quality for the two or more digital video streams is determined by assessing the video quality of at least one of the two or more digital video streams at a greater importance than another digital video stream in the two or more digital video streams.

17. The apparatus of claim 10 , wherein the execution of the one or more sequences of instructions further causes:

dynamically increasing some portion of the compute capacity allocated to encode the particular digital video stream based on, at least in part, the determined video complexity information, wherein dynamically increasing comprising:

assessing a first impact of decreasing a first portion of the compute capacity allocated to encode at least one digital video stream of the two or more digital video streams;

assessing a second impact of increasing a second portion of the compute capacity to encode the particular digital video stream proportional to the first portion.

18. The apparatus of claim 10 , wherein at least two of the two or more video streams are not encoded on the same physical machine.

19. A method for dynamically allocating compute capacity for use in encoding a video stream, comprising:

upon the occurrence of each periodic interval in a series of periodic intervals, performing: (a) determining, for a particular future periodic interval of said series of periodic intervals, video complexity information for two or more digital video streams actively being encoded at the same time, (b) determining whether an amount of the compute capacity allocated for encoding a particular digital video stream of the two or more digital video streams should be adjusted during said particular future periodic interval of said series of periodic intervals based on (i) the determined video complexity information and (ii) an assessment of impact to quality of said particular digital video stream resulting from making said adjustment to said amount of compute capacity, and (c) upon determining that the amount of compute capacity currently allocated for encoding said particular digital video stream of the two or more digital video streams should be adjusted during said future periodic interval, adjusting the amount of compute capacity currently allocated during said future periodic interval to encoding the particular digital video stream of the two or more digital video streams.

20. The method of claim 19 , wherein the determined video complexity information describes the complexity of digital video carried by the two or more digital video streams across a bounded number of consecutive digital frames, wherein said bounded number of consecutive digital frames includes digital frames not yet encoded.

21. The method of claim 19 , further comprising:

dynamically adjusting a portion of the compute capacity allocated for encoding the particular digital video stream by maximizing a measure of optimal video quality calculated for the two or more digital video streams using, at least in part, the determined video complexity information.

22. The method of claim 21 , wherein maximizing the measure of optimal video quality for the two or more digital video streams comprises:

identifying one or more features in at least one of the two or more digital video streams in which support for will be reduced or ceased during said future periodic interval.

23. The method of claim 21 , wherein the measure of optimal video quality for the two or more digital video streams is determined by assessing the video quality of at least one of the two or more digital video streams at a greater importance than another digital video stream in the two or more digital video streams.

24. The method of claim 19 , further comprising:

dynamically increasing some portion of the compute capacity allocated to encode the particular digital video stream based on, at least in part, the determined video complexity information, wherein dynamically increasing comprising:

assessing a first impact of decreasing a first portion of the compute capacity allocated to encode at least one digital video stream of the two or more digital video streams;

assessing a second impact of increasing a second portion of the compute capacity to encode the particular digital video stream proportional to the first portion.

Assignments (2)
SECURITY INTEREST Recorded Dec 21, 2023
From: HARMONIC INC.
To: CITIBANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 066090/0906 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2022
From: HALTMAYER, NEVEN
To: HARMONIC, INC.
Reel/Frame 060549/0707 →