Granted Patent B2

US 12,688,148 · App. 18/794,143 · Granted Jul 21, 2026

Inventors: Greg Dykema (Palo Alto, CA); Aarti Lalwani (Palo Alto, CA)

Assignee: SambaNova Systems, Inc.

G06F15/825G06F13/4068

Abstract

Claims (42)

1 . A data processing system, comprising:

a first computing system comprising a first processor and a topological communications configurator (TCC) comprising a computing program configured to execute on the first processor;

a second computing system comprising a plurality of compute nodes, compute nodes among the plurality of compute nodes comprising at least one dataflow processor, each of the at least one dataflow processor configured to execute a compute worker among a plurality of compute workers, the plurality of compute workers configured to execute a computing application of the second computing system; and

an internodal fabric configured to communicatively couple compute nodes among the plurality of compute nodes,

wherein the TCC is configured to:

determine an internodal interconnection topology of the plurality of compute nodes, the internodal interconnection topology comprising the internodal fabric;

determine, based on the internodal interconnection topology, internodal communications routes communicatively interconnecting, via the internodal fabric, a set of intranodal segments among a plurality of intranodal segments, each of the plurality of intra nodal segments comprising an intranodal interconnection of dataflow processors included in respective nodes among the plurality of compute nodes, each of the plurality of intranodal segments comprising a set of compute workers, among the plurality of compute workers, executed by the dataflow processors included in a node among the respective nodes, the set of compute workers included in a worker logical topology;

determine internodal communications costs corresponding to communications routes among the internodal communications routes; and

determine, based on the internodal communications costs, a connections-optimized configuration of interconnected segments among the set of intranodal segments.

2 . The data processing system of claim 1 , wherein the TCC is further configured to determine, based on the internodal communications costs, a cost-optimized interconnection topology of the interconnected segments.

3 . The data processing system of claim 2 , wherein the cost-optimized interconnection topology of the interconnected segments is a ring topology.

4 . The data processing system of claim 1 , wherein a first segment, among the plurality of intranodal segments, comprises a head dataflow processor;

wherein a second segment, among the plurality of intranodal segments, comprises a tail dataflow processor; and

wherein the TCC is further configured to determine, based on the internodal communications costs, a cost-optimized interconnection of the head dataflow processor, included in the first segment, and the tail dataflow processor included in the second segment.

5 . The data processing system of claim 1 , wherein the TCC is further configured to determine a connections-optimized configuration of a set of dataflow processors of a node among the plurality of compute nodes, the set of dataflow processors included in a segment, among the plurality of intranodal segments included in the node.

6 . The data processing system of claim 1 , wherein the TCC is further configured to determine the connections-optimized configuration of dataflow processors included in the segment by

determining an intranodal interconnection topology of the node, the intranodal interconnection topology comprising interconnections of the set of dataflow processors via an intranodal fabric;

determining, based on the intranodal interconnection topology, a set of intranodal communications routes communicatively interconnecting, via the intranodal fabric, the set of dataflow processors;

determining intranodal communications costs corresponding to communications routes among the set of intranodal communications routes; and

determining based on the intranodal communications costs, a connections-optimized configuration of the set of dataflow processors.

7 . The data processing system of claim 1 , wherein the set of compute workers is configured to execute operations of the computing application as a pipeline of compute workers.

8 . The data processing system of claim 1 , wherein the TCC configured to determine the internodal communications costs comprises based on performance characteristics selected from a group consisting of performance characteristics of the internodal fabric, and performance characteristics of an interconnect coupling a first segment among the plurality of intranodal segments.

9 . The data processing system of claim 8 , wherein a performance characteristic among the performance characteristics of the internodal fabric is selected from a group consisting of performance characteristics of the internodal fabric, performance characteristics of the interconnect coupling the first segment and the internodal fabric, and performance characteristics associated with a physical locality of the internodal fabric within the second computing system.

10 . The data processing system of claim 8 , wherein a performance characteristic among the performance characteristics of the interconnect is selected from a group consisting of a utilization of the interconnect, a throughput of the interconnect, a data rate of the interconnect, a communications latency of the interconnect, and a physical locality of the interconnect within the second computing system.

11 . A computer-implemented method comprising:

determining, by a topological communications configurator (TCC) of a first computing system, an internodal interconnection topology of a plurality of compute nodes of a second computing system, the internodal interconnection topology comprising an internodal fabric;

determining, by the TCC, based on the internodal interconnection topology, internodal communications routes interconnecting, via the internodal fabric, a set of intranodal segments among a plurality of intranodal segments, each of the plurality of intranodal segments comprising an intranodal interconnection of dataflow processors included in respective nodes among the plurality of compute nodes, each of the plurality of intranodal segments corresponding to a respective portion of a worker logical topology comprising compute workers configured to execute an application of the second computing system;

determining, by the TCC, internodal communications costs corresponding to communications routes among the internodal communications routes; and

determining a connections-optimized configuration of interconnected segments.

12 . The method of claim 11 , wherein the method of determining the connections-optimized configuration of integrated segments comprises the TCC determining, based on the internodal communications costs, a cost-optimized interconnection of the interconnected segments to form a ring topology among the interconnected segments.

13 . The method of claim 11 , wherein a first segment and a second segment, among the plurality of intranodal segments, each comprise a head and a tail dataflow processor; and

wherein the method determining, by the TCC, the connections-optimized configuration of integrated segments comprises the TCC determining, further based on the internodal communications costs, a cost-optimized interconnection of the tail dataflow processor of the first segment and a head dataflow processor of the second segment.

14 . The method of claim 11 , the method further comprising the TCC determining a connections-optimized configuration of dataflow processors included in a first segment of a first node, the first segment among the plurality of intranodal segments, the first node among the plurality of compute nodes.

15 . The method of claim 11 , wherein the compute workers comprise data-parallel workers configured to execute operations of the application, on dataflow processors among the dataflow processors included in the respective nodes among the plurality of compute nodes, as a pipeline.

16 . The method of claim 11 , wherein the internodal communications costs are based on performance characteristics selected from a group consisting of performance characteristics of the internodal fabric and performance characteristics of an interconnect coupling a first segment among the plurality of intranodal segments and the internodal fabric.

17 . The method of claim 16 , wherein a performance characteristic among the performance characteristics of the internodal fabric is selected from a group consisting of a utilization of the internodal fabric; a throughput of a communications route through the internodal fabric; a latency of a communications route through the internodal fabric; and a physical locality of the internodal fabric within the second computing system.

18 . The method of claim 16 , wherein a performance characteristic among the performance characteristics of the interconnect is selected from a group consisting of a utilization of the interconnect; a throughput of the interconnect; a data rate of the interconnect; a communications latency of the interconnect; and a physical locality of the interconnect within the second computing system.

19 . A computer program product comprising a computer readable storage medium having first program instructions embodied therewith, wherein the first program instructions are executable by at least one processor to cause the at least one processor to:

determine an internodal interconnection topology of a plurality of compute nodes of a computing system, the internodal interconnection topology comprising an internodal fabric;

determine, based on the internodal interconnection topology, internodal communications routes communicatively interconnecting, via the internodal fabric, a set of intranodal segments among a plurality of intranodal segments, each of the plurality of intranodal segments comprising an intranodal interconnection of processors included in respective nodes among the plurality of compute nodes, each of the plurality of intranodal segments corresponding to a respective portion of a worker logical topology comprising compute workers configured to execute an application of the computing system;

determine internodal communications costs corresponding to communications routes among the internodal communications routes; and

determine, based on the internodal communications costs, a connections-optimized configuration of interconnected segments.

Assignments (2)

Continuity (3)

References Cited (31)