IP Library Granted Patent US 10,719,970
Granted Patent B2
US 10,719,970 · App. 15/864,833 · Granted Jul 21, 2020

Low latency firmware command selection using a directed acyclic graph

Inventors: Kutty Banerjee (San Jose, CA); Rohan Sanjeev Patil (San Francisco, CA); Pratik Chandresh Shah (Santa Clara, CA); Gokhan Avkarogullari (San Jose, CA); Tatsuya Iwamoto (Foster City, CA)
Assignee: Apple Inc.
G06T15/005G06F9/4881G06T15/80G09G5/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,719,970
App. No.
15/864,833
Granted
Jul 21, 2020
Kind
B2
Abstract

One disclosed embodiment includes a method of scheduling graphics commands for processing. A plurality of micro-commands is generated based on one or more graphics commands obtained from a central processing unit. The dependency between the one or more graphics commands is then determined and an execution graph is generated based on the determined dependency. Each micro-command in the execution graph is connected by an edge to the other micro-commands that it depends on. A wait count is defined for each micro-command of the execution graph, where the wait count indicates the number of micro-commands that the each particular micro-command depends on. One or more micro-commands with a wait count of zero are transmitted to a ready queue for processing.

Claims (64)

1. A method of low-latency graphics processing, comprising:

obtaining, from a central processing unit (CPU), a plurality of graphics commands;

generating, based on the plurality of graphics commands, a plurality of micro-commands including pre-processing micro-commands to retrieve data, kick micro-commands to process the retrieved data, and post-processing micro-commands to store processed data;

determining dependencies between the plurality of micro-commands and using a data structure to maintain a registry of the plurality of micro-commands and the determined dependencies;

creating an execution graph separate from the data structure based on the determined dependencies in the data structure, wherein each micro-command in the execution graph is represented as a node and wherein dependencies in the execution graph are represented as edges between related micro-commands;

defining a wait count for each micro-command within the execution graph, wherein the wait count for a particular micro-command is a number of micro-commands that the particular micro-command depends on; and

transmitting one or more micro-commands with the wait count of zero to a ready queue for processing.

2. The method of claim 1 , wherein determining comprises:

registering the plurality of micro-commands in a data structure, wherein the data structure tracks the dependency between the plurality of micro-commands.

3. The method of claim 2 , wherein the data structure comprises a first hash table having an entry for each micro-command stored in the execution graph, each entry including a tuple identifying all of the micro-commands that depend on the entry's corresponding micro-command.

4. The method of claim 1 , further comprising:

determining, for the plurality of micro-commands, one or more priority categories, wherein each of the micro-commands of the plurality of micro-commands is associated with one of the one or more priority categories.

5. The method of claim 4 , further comprising:

receiving a priority policy to define priorities of the one or more priority categories, wherein the execution graph is further created based on the priority policy.

6. The method of claim 1 , further comprising:

updating the wait count for a first group of micro-commands within the execution graph after completion of processing of one or more transmitted micro-command that the first group of micro-commands depend upon.

7. The method of claim 1 , wherein the execution graph comprises a directed acyclic graph.

8. The method of claim 1 ,

wherein the pre-processing micro-commands are associated with a pre-processing stage,

wherein the kick micro-commands are associated with a kick stage,

wherein the post-processing micro-commands are associated with a post-processing stage, and

wherein the kick stage has a different execution priority relative to the pre-processing stage and the post-processing stage.

9. A method of low-latency graphics processing, comprising:

analyzing an execution graph comprising a plurality of micro-commands, wherein the execution graph is generated from a data structure registry of micro-commands and related dependencies separate from the execution graph, wherein the plurality of micro-commands include pre-processing micro-commands to retrieve data, kick micro-commands to process the retrieved data, and post-processing micro-commands to store the processed data, wherein the execution graph includes nodes that represent each micro-command and edges that represent dependencies between respective micro-commands;

identifying, based on the analysis, at least one micro-command for processing;

storing the at least one micro-command in a ready queue for processing;

receiving a first signal indicative that the at least one micro-command has completed processing;

determining, in response to the first signal, one or more other micro-commands in the execution graph that are dependent on the at least one micro-command; and

updating each identified dependent micro-command to reflect the at least one micro-command has finished processing.

10. The method of claim 9 , wherein identifying further comprises:

selecting at least one micro-command having a wait count of zero, wherein the wait count represents a number of micro-commands the at least one micro-command depends on.

11. The method of claim 10 , wherein updating comprises:

decrementing the wait count of each identified dependent command.

12. The method of claim 9 , wherein storing comprises:

storing the at least one micro-command in one of a pre-processing queue, a kick queue, and a post-processing queue.

13. The method of claim 9 , wherein the execution graph comprises a directed acyclic graph.

14. The method of claim 9 , further comprising determining a priority of the at least one micro-command.

15. The method of claim 14 , further comprising selecting a first micro-command from the ready queue for processing based on a priority of the first micro-command.

16. The method of claim 9 ,

wherein the pre-processing micro-commands are associated with a pre-processing stage,

wherein the kick micro-commands are associated with a kick stage,

wherein the post-processing micro-commands are associated with a post-processing stage, and

wherein the kick stage has a different execution priority relative to the pre-processing stage and the post-processing stage.

17. A non-transitory computer readable medium comprising instructions stored thereon to support graphics processing; the instructions when executed cause one or more processors to:

obtain, from a central processing unit (CPU), a plurality of graphics commands;

generate, based on the plurality of graphics commands, a plurality of micro-commands including pre-processing micro-commands to retrieve data, kick micro-commands to process the retrieved data, and post-processing micro-commands to store the processed data;

determining dependencies between the plurality of micro-commands and using a data structure to maintain a registry of the plurality of micro-commands and the determined dependencies;

creating an execution graph separate from the data structure based on the determined dependencies in the data structure, wherein each micro-command in the execution graph is represented as a node and wherein dependencies in the execution graph are represented as edges between related micro-commands;

define a wait count for each micro-command within the execution graph, wherein the wait count for a particular micro-command is a number of micro-commands that the particular micro-command depends on; and

transmit one or more micro-commands with the wait count of zero to a ready queue for processing.

18. The non-transitory computer readable medium of claim 17 , wherein the instructions to cause the one or more processers to determine, further comprises instructions to cause one or more processor to:

register the plurality of micro-commands in a data structure, wherein the data structure tracks the dependency between the plurality of micro-commands.

19. The non-transitory computer readable medium of claim 18 , wherein the data structure comprises a first hash table having an entry for each micro-command stored in the execution graph, each entry including a tuple identifying all of the micro-commands that depend on the entry's corresponding micro-command.

20. The non-transitory computer readable medium of claim 17 , further comprises instructions to cause the one or more processers to:

determine, for the plurality of micro-commands, one or more priority categories, wherein each of the micro-commands of the plurality of micro-commands is associated with one of the one or more priority categories.

21. The non-transitory computer readable medium of claim 20 , further comprises instructions to cause the one or more processers to:

receive a priority policy to define priorities of the one or more priority categories, wherein the execution graph is further created based on the priority policy.

22. The non-transitory computer readable medium of claim 17 , further comprises instructions to cause the one or more processers to:

update the wait count for a first group of micro-commands within the execution graph after completion of processing of one or more transmitted micro-commands that the first group of micro-commands depend upon.

23. The non-transitory computer readable medium of claim 17 ,

wherein the pre-processing micro-commands are associated with a pre-processing stage,

wherein the kick micro-commands are associated with a kick stage,

wherein the post-processing micro-commands are associated with a post-processing stage, and

wherein the kick stage has a different execution priority relative to the pre-processing stage and the post-processing stage.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2018
From: BANERJEE, KUTTY; PATIL, ROHAN SANJEEV; SHAH, PRATIK CHANDRESH; AVKAROGULLARI, GOKHAN; IWAMOTO, TATSUYA
To: APPLE INC.
Reel/Frame 045319/0518 →
Continuity (1)
Related Publication 20190213776A1 · Jul 11, 2019