Dynamically reducing latency in processing pipelines

1. A method comprising:

obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages used to process a plurality of frames in parallel, wherein the plurality of frames are processed at each of the plurality of execution stages in a sequential order, starting with an initial execution stage and continuing with each subsequent execution stage;

determining a first largest weighted average execution time associated with one of the plurality of execution stages; and

applying a delay to the initial execution stage prior to processing a first next frame, wherein the delay is substantially equivalent to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial execution stage of the plurality of execution stages.

2. The method of claim 1 , further comprising:

updating, upon processing the first next frame in at least one execution stage of the plurality of execution stages, an weighted average execution time of the at least one execution stage of the plurality of execution stages;

determining a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and

updating the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

3. The method of claim 1 , further comprising:

adjusting, prior to processing subsequent frames to a third next frame, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

4. The method of claim 3 , wherein adjusting the delay comprises:

determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and

increasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

5. The method of claim 3 , wherein adjusting the delay comprises:

determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is less than zero; and

decreasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

6. The method of claim 5 , wherein the predefined minimum value indicates there is no delay or queue between processing of the second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first subsequent frame at the execution stage associated with the first largest weighted average execution time processing.

7. The method of claim 3 , wherein adjusting the delay comprises:

receiving, using a proportional-integral-derivative (PID) controller, the period of time between processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first next frame at the execution stage associated with the first largest weighted average execution time; and

outputting, using the PID controller, an adjustment value to be applied to the delay based on the period of time matching a predefined minimum value; and

applying the adjustment value to the delay.

8. A system comprising:

one or more processing units to:

obtain an weighted average execution time associated with each execution stage of a plurality of execution stages used to process a plurality of frames in parallel, wherein the plurality of frames are processed at each of the plurality of execution stages in a sequential order, starting with an initial execution stage and continuing with each subsequent execution stage;

determine a first largest weighted average execution time associated with one of the plurality of execution stages; and

apply a delay to the initial execution stage prior to processing a first next frame, wherein the delay is substantially equivalent to a difference between the first largest weighted average execution time and an execution time of a previous frame in the initial execution stage of the plurality of execution stages.

9. The system of claim 8 , wherein the one or more processing units further:

update, upon processing the first next frame in at least one execution stage of the plurality of execution stages, an weighted average execution time of the at least one execution stage of the plurality of execution stages;

determine a second largest weighted average execution time in view of the updated weighted average execution time of the at least one execution stage; and

update the delay to be applied to the initial execution stage based on the second largest weighted average execution time.

10. The system of claim 8 , wherein the one or more processing units further:

adjust, prior to processing subsequent frames to a third next frame, the delay to be applied to the initial execution stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time.

11. The system of claim 10 , wherein to adjust the delay, the one or more processing units further:

determine that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of the first next frame at the execution stage associated with the first largest weighted average execution time is greater than zero; and

increase the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

12. The system of claim 10 , wherein to adjust the delay, the one or more processing units further:

determine that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of the first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is less than zero; and

decrease the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time is equal to a predefined minimum value.

13. The system of claim 12 , wherein the predefined minimum value indicates that there is no delay or queue between processing of a second subsequent frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of a first subsequent frame at the execution stage associated with the first largest weighted average execution time processing.

14. The system of claim 13 , wherein to adjust the delay, the one or more processing units further:

receive, using a proportional-integral-derivative (PID) controller, the period of time between processing of the second next frame at the execution stage preceding the execution stage associated with the first largest weighted average execution time and processing of the first next frame at the execution stage associated with the first largest weighted average execution time; and

output, using the PID controller, an adjustment value to be applied to the delay based on the period of time matching a predefined minimum value, and applying the adjustment value to the delay.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

obtaining a weighted average execution time associated with each execution stage of a plurality of execution stages used to process a plurality of frames in parallel, wherein the plurality of frames are processed at each of the plurality of execution stages in a sequential order;

applying a delay to an initial stage of the plurality of execution stages based on a largest weighted average execution time among the weighted average execution time associated with the plurality of execution stages; and

adjusting the delay to the initial stage based on a period of time between (i) processing of a second next frame at an execution stage preceding an execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time, wherein the delay is substantially equivalent to a difference between the largest weighted average execution time among the weighted average execution time associated with the plurality of execution stages and an execution time of a previous frame in the initial stage.

16. The non-transitory computer-readable storage medium of claim 15 , wherein adjusting the delay includes determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of the first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is greater than zero, and increasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is equal to a predefined minimum value.

17. The non-transitory computer-readable storage medium of claim 15 , wherein adjusting the delay includes determining that the period of time between (i) processing of the second next frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of the first next frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is less than zero, and decreasing the delay until the period of time between (i) processing of a second subsequent frame at the execution stage preceding the execution stage of the plurality of execution stages associated with the largest weighted average execution time and (ii) processing of a first subsequent frame at the execution stage of the plurality of execution stages associated with the largest weighted average execution time is equal to a predefined minimum value.