Granted Patent B2

US 12,333,283 · App. 18/129,718 · Granted Jun 17, 2025

Inventors: Blaine Rister (San Francisco, CA); Haocheng Dong (San Jose, CA); David Alan Koeplinger (Egg Harbor, NJ); Yaqi Zhang (Foster City, CA); Junjue Wang (Newark, CA); Zhuo Chen (Palo Alto, CA); Arvind Sujeeth (Palo Alto, CA)

Assignee: SambaNova Systems, Inc.

G06F8/433

Abstract

Claims (65)

1. A method, the method comprising:

performing, by a compiler included in a first computing system, a trial compilation of a first high level decision to a first LLIR (low level intermediate representation), the first high level decision among high level decisions to execute a dataflow application on a second computing system, the first LLIR comprising first hardware resources of the second computing system to execute the dataflow application based on the first high level decision;

determining, by the compiler, first low level execution metrics associated with the first hardware resources included, by the trial compilation of the first high level decision, in the first LLIR;

determining, by the compiler, based on the first low level execution metrics, a first trial result;

performing, by the compiler, a trial compilation of a second high level decision, among the high level decisions, to a second LLIR comprising second hardware resources of the second computing system to execute the dataflow application based on the second high level decision;

determining, by the compiler, second low level execution metrics associated with the second hardware resources included, by the trial compilation of the second high level decision, in the second LLIR;

determining, by the compiler, based on the second low level execution metrics, a second trial result;

evaluating, by the compiler, based on a first execution objective, the first trial result;

evaluating, by the compiler, based on a second execution objective, the second trial result; and,

selecting, by the compiler, based on the evaluating the first trial result and the evaluating the second trial result, at least one of the first high level decision and the second high level decision for executing the dataflow application on the second computing system.

2. The method of claim 1 , wherein the evaluating the first trial result based on the first execution objective comprises comparing a metric among the first low level execution metrics to an application execution objective.

3. The method of claim 2 , wherein the first execution objective is selected from a group consisting of: increasing utilization of third hardware resources of the second computing system to execute the dataflow application; increasing parallelism of fourth hardware resources of the second computing system to execute the dataflow application; and, decreasing execution latency of the second computing system to execute the dataflow application.

4. The method of claim 1 , the method further comprising determining, by the compiler, based on the evaluating the first trial result, a modification of the first high level decision.

5. The method of claim 4 , the method further comprising:

performing, by the compiler, a trial compilation of the modification of the first high level decision to a third LLIR comprising third hardware resources of the second computing system to execute the dataflow application based on the modification of the first high level decision;

determining, by the compiler, third low level execution metrics associated with the third hardware resources included, by the trial compilation of the modification of the first high level decision, in the third LLIR;

determining, by the compiler, based on the third low level execution metrics, a third trial result; and,

evaluating the third trial result, by the compiler, based on a third execution objective associated with executing the dataflow application on the second computing system.

6. The method of claim 1 , the method further comprising discarding, by the compiler, based on the evaluating the first trial result and the evaluating the second trial result, at least one of the first high level decision and the second high level decision from among the high level decisions to execute the dataflow application on the second computing system.

7. The method of claim 1 , wherein the first high level decision corresponds to executing a section cut included in an HLR of the dataflow application.

8. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the program instructions are executable by at least one processor of a first computing system to cause the at least one processor to:

perform a trial compilation of a first high level decision to a first LLIR (low level intermediate representation), the first high level decision among high level decisions to execute a dataflow application on a second computing system, the first LLIR comprising first hardware resources of the second computing system to execute the dataflow application based on the first high level decision;

determine first low level execution metrics associated with the first hardware resources included, by the trial compilation of the first high level decision, in the first LLIR;

determine, based on the first low level execution metrics, a first trial result;

perform a trial compilation of a second high level decision, among the high level decisions, to a second LLIR comprising second hardware resources of the second computing system to execute the dataflow application based on the second high level decision;

determine second low level execution metrics associated with the second hardware resources included, by the trial compilation of the second high level decision, in the second LLIR;

determine, based on the second low level execution metrics, a second trial result;

evaluate, based on a first execution objective, the first trial result;

evaluate, based on a second execution objective, the second trial result; and,

select, based on the evaluating the first trial result and the evaluating the second trial result, at least one of the first high level decision and the second high level decision for executing the dataflow application on the second computing system.

9. The computer program product of claim 8 , wherein the program instructions are executable by the at least one processor to further cause the at least one processor to:

determine, based on evaluating the first trial result, a modification of the first high level decision;

perform, a trial compilation of the modification of the first high level decision to a third LLIR comprising third hardware resources of the second computing system to execute the dataflow application based on the modification of the first high level decision;

determine third low level execution metrics associated with the third hardware resources included, by the trial compilation of the modification of the first high level decision, in the third LLIR;

determine, based on the third low level execution metrics, a third trial result; and,

evaluate, based on a third execution objective associated with executing the dataflow application on the second computing system.

10. The computer program product of claim 8 , wherein the program instructions are executable by the at least one processor to further cause the at least one processor to discard, based on the evaluating the first trial result and the evaluating the second trial result, at least one of the first high level decision and the second high level decision from among the high level decisions to execute the dataflow application on the second computing system.

11. A first computing system comprising: one or more processors, and

an HLR (high level representation) of a dataflow application;

an HL (high level) compiler executable on a first processor of the first computing system, wherein the HL compiler is configured to determine, based on the HLR, high level decisions to execute the dataflow application on a second computing system; and,

an LL (low level) compiler executable on a second processor of the first computing system, wherein the LL compiler is configured to:

perform a trial compilation of a first high level decision, among the high level decisions, to a first LLIR (low level intermediate representation), the first LLIR comprising first hardware resources of the second computing system to execute the dataflow application based on the first high level decision;

determine first low level execution metrics associated with the first hardware resources included, by the trial compilation of the first high level decision, in the first LLIR;

determine, based on the first low level execution metrics, a first trial result;

perform a trial compilation of a second high level decision, among the high level decisions, to a second LLIR comprising second hardware resources of the second computing system to execute the dataflow application based on the second high level decision;

determine second low level execution metrics associated with the second hardware resources included, by the trial compilation of the second high level decision, in the second LLIR;

determine, based on the second low level execution metrics, a second trial result;

and, communicate, to the HL compiler, the first trial result and the second trial result; wherein the HL compiler is further configured to:

evaluate, based on a first execution objective, the first trial result;

evaluate, based on a second execution objective, the second trial result; and,

select, based on the evaluating the first trial result and the evaluating the second trial result, at least one of the first high level decision and the second high level decision for executing the dataflow application on the second computing system.

12. The first computing system of claim 11 , wherein the LL compiler configured to evaluate the first trial result based on the first execution objective comprises the LL compiler further configured to evaluate the first trial result by comparing a metric among the first low level execution metrics to a first application execution objective; and,

wherein the LL compiler configured to evaluate the second trial result based on the second execution objective comprises the LL compiler further configured to evaluate the second trial result by comparing a metric among the second low level execution metrics to a second application execution objective.

13. The first computing system of claim 11 , wherein the computing system further comprises a hardware specification comprising a specification of the first hardware resources and the second hardware resources; and,

wherein the LL compiler configured to determine the first low level execution metrics and to determine the second low level execution metrics comprises LL compiler configured to determine the first low level execution metrics and to determine the second low level execution metrics based on the hardware specification.

14. The first computing system of claim 11 , wherein the HL compiler is further configured to determine, based on evaluating the first trial result, a modification of the first high level decision.

15. The first computing system of claim 14 , wherein the LL compiler is further configured to: perform a trial compilation of the modification of the first high level decision to a third LLIR comprising third hardware resources of the second computing system to execute the dataflow application based on the modification of the first high level decision;

determine third low level execution metrics associated with the third hardware resources included, by the trial compilation of the modification of the first high level decision, in the third LLIR;

determine, based on the third low level execution metrics, a third trial result; and,

wherein the HL compiler is further configured to evaluate the third trial result based on a third execution objective associated with executing the dataflow application on the second computing system.

16. The first computing system of claim 11 , wherein the HL compiler is further configured to discard, based on evaluating the first trial result and evaluating the second trial result, at least one of the first high level decision and the second high level decision from among the high level decisions to execute the dataflow application on the second computing system.

17. The first computing system of claim 11 , wherein and the first high level decision is associated with executing a section cut included in the HLR.

18. The first computing system of claim 11 , wherein the HL compiler configured to evaluate the first trial result based on the first execution objective comprises the HL compiler further configured to evaluate the first trial result by comparing a metric, among the first low level execution metrics, to an application execution objective.

19. The first computing system of claim 11 , wherein the first execution objective is selected from a group consisting of: increasing utilization of third hardware resources of the second computing system to execute the dataflow application; increasing parallelism of fourth hardware resources of the second computing system to execute the dataflow application; and, decreasing execution latency of the second computing system to execute the dataflow application.

20. The first computing system of claim 11 , wherein a metric among the first low level execution metrics is selected from a group consisting of: a number of processors among the first hardware resources to execute the dataflow application based on the first high level decision; a number of parallel computations to execute the dataflow application based on the first high level decision; an execution latency of a processor among the first hardware resources to execute the dataflow application based on the first high level decision; a size a memory among the first hardware resources to execute the dataflow application based on the first high level decision; and, a data transfer latency among memories of the first hardware resources to execute the dataflow application based on the first high level decision.

Assignments (2)

Continuity (8)

References Cited (25)