Dynamically reducing latency in processing pipelines

1. A method comprising:

obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages processing a plurality of frames starting with an initial execution stage and continuing with each subsequent execution stage;

determining a first largest weighted average execution time associated with one of the plurality of execution stages; and

applying a delay to the initial execution stage prior to processing a first next frame of the plurality of frames, wherein the delay is based on the first largest weighted average execution time of the plurality of execution stages.

2. The method of claim 1 , further comprising:

updating, upon processing the first next frame in at least one execution stage of the plurality of execution stages, a weighted average execution time of the at least one execution stage of the plurality of execution stages;

determining a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and

updating the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

3. The method of claim 1 , wherein the delay is equal to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial execution stage of the plurality of execution stages.

4. The method of claim 1 , further comprising:

adjusting, prior to processing subsequent frames, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

5. The method of claim 4 , wherein adjusting the delay comprises:

determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and

increasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

6. The method of claim 4 , wherein adjusting the delay comprises:

determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is less than zero; and

decreasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

7. The method of claim 6 , wherein the predefined minimum value indicates there is no delay or queue between processing of the second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first subsequent frame at the execution stage associated with the first largest weighted average execution time processing.

8. The method of claim 4 , wherein adjusting the delay comprises:

receiving, using a proportional-integral-derivative (PID) controller, the period of time between processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first next frame at the execution stage associated with the first largest weighted average execution time; and

outputting, using the PID controller, an adjustment value to be applied to the delay based on the period of time matching a predefined minimum value; and

applying the adjustment value to the delay.

9. A system comprising:

one or more processing units to:

obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages processing a plurality of frames starting with an initial execution stage and continuing with each subsequent execution stage;

determining a first largest weighted average execution time associated with one of the plurality of execution stages; and

applying a delay to the initial execution stage prior to processing a first next frame of the plurality of frames, wherein the delay is based on the first largest weighted average execution time of the plurality of execution stages.

10. The system of claim 9 , wherein the one or more processing units further:

update, upon processing the first next frame in at least one execution stage of the plurality of execution stages, a weighted average execution time of the at least one execution stage of the plurality of execution stages;

determine a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and

update the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

11. The system of claim 9 , wherein the delay is equal to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial executions stage of the plurality of execution stages.

12. The system of claim 9 , wherein the one or more processing units further:

adjust, prior to processing subsequent frames, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

13. The system of claim 12 , wherein to adjust the delay, the one or more processing units further:

determine that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and

increase the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

14. The system of claim 12 , wherein to adjust the delay, the one or more processing units further:

determine that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of the first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is less than zero; and

decrease the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

15. The system of claim 14 , wherein the predefined minimum value indicates that there is no delay or queue between processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time processing.

16. The system of claim 15 , wherein to adjust the delay, the one or more processing units further:

receive, using a proportional-integral-derivative PID controller, the period of time between processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first next frame at the execution stage associated with the first largest weighted average execution time; and

output, using the PID controller, an adjustment value to be applied to the delay based on the period of time matching a predefined minimum value, and applying the adjustment value to the delay.

17. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages processing a plurality of frames;

applying a delay to an initial stage of the plurality of execution stages based on a largest weighted average execution time associated with one of the plurality of execution stages; and

adjusting the delay to the initial stage based on a period of time between processing in sequentially adjacent execution stages.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the delay is equal to a difference between the largest weighted average execution time among the weighted average execution times associated with the plurality of execution stages and an execution time of a previous frame in the initial stage.

19. The non-transitory computer-readable storage medium of claim 17 , wherein adjusting the delay comprises:

determining that the period of time is a first period of time between (i) processing of a second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is greater than zero;

providing, to a proportional-integral-derivative (PID) controller, the first period of time to increase the delay until a second period between (i) processing of a second subsequent frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is equal to a predefined minimum value.

20. The non-transitory computer-readable storage medium of claim 17 , wherein adjusting the delay comprises:

determining that the period of time is a first period of time between (i) processing of a second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is less than zero;

providing, to a proportional-integral-derivative (PID) controller, the period of time to decrease the delay until a second period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is equal to a predefined minimum value.