Granted Patent B2

US 12,602,349 · App. 18/213,598 · Granted Apr 14, 2026

Inventors: Abhishek Srivastava (Sunnyvale, CA); Matthew Vilim (Redwood City, CA); Raghu Prabhakar (San Jose, CA); Sankar Rachuru (Palo Alto, CA); Zhekun Zhang (Palo Alto, CA); Matheen Musaddiq (Austin, TX); Apurv Vivek (Palo Alto, CA); Sitanshu Gupta (Palo Alto, CA)

Assignee: SambaNova Systems, Inc.

G06F15/7867G06F15/825

Abstract

Claims (84)

1 . A computer-implemented method to execute a program on a coarse-grained reconfigurable (CGR) processor comprising an array of CGR units, the computer-implemented method comprising:

determining that the program produces an output that includes a variable length tensor;

determining a maximum size of the variable length tensor;

setting, based on the maximum size, a maximum value of a counter associated with the program;

setting a counter value of the counter to an initial value of zero;

initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;

determining that the program is operating on a first portion of the input tensor;

performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and

communicating the updated counter value to one or more consumers of the counter within the program.

2 . The computer-implemented method of claim 1 , wherein:

determining that the program produces the output that includes a variable length tensor is performed at a compile time.

3 . The computer-implemented method of claim 1 , wherein:

determining the maximum size of the variable length tensor is performed at a compile time.

4 . The computer-implemented method of claim 1 , further comprising:

determining that the program is operating on a second portion of the input tensor;

performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and

communicating the second updated counter value to the one or more consumers of the counter within the program.

5 . The computer-implemented method of claim 1 , further comprising:

determining that the program has completed operating on the input tensor;

determining a final size of the output; and

communicating the final size to one or more downstream consumers of the output.

6 . The computer-implemented method of claim 1 , wherein the counter is located in:

a read context of a memory unit that is used when reading the first portion of the input tensor; or

a write context of the memory unit that is used when writing a portion of the output.

7 . The computer-implemented method of claim 6 , wherein:

the write context performs a write operation to a first buffer in a scratchpad of the memory unit;

the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and

the read operation and the write operation overlap in time.

8 . A non-transitory computer-readable storage medium storing computer program instructions that, when executed on a processor, perform operations comprising:

determining that a program produces an output that includes a variable length tensor;

determining a maximum size of the variable length tensor;

setting, based on the maximum size, a maximum value of a counter associated with the program;

setting a counter value of the counter to an initial value of zero;

initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;

determining that the program is operating on a first portion of the input tensor;

performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and

communicating the updated counter value to one or more consumers of the counter within the program.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein:

determining that the program produces the output that includes a variable length tensor is performed at compile time.

10 . The non-transitory computer-readable storage medium of claim 8 , wherein:

determining the maximum size of the variable length tensor is performed at compile time.

11 . The non-transitory computer-readable storage medium of claim 8 , the operations further comprising:

determining that the program is operating on a second portion of the input tensor;

performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and

communicating the second updated counter value to the one or more consumers of the counter within the program.

12 . The non-transitory computer-readable storage medium of claim 8 , the operations further comprising:

determining that the program has completed operating on the input tensor;

determining a final size of the outputs; and

communicating the final size to one or more downstream consumers of the output of the program.

13 . The non-transitory computer-readable storage medium of claim 8 , wherein the counter is located in:

a read context of a memory unit that is used when reading the first portion of the input tensor; or

a write context of the memory unit that is used when writing a portion of the output.

14 . The non-transitory computer-readable storage medium of claim 13 , wherein:

the write context performs a write operation to a first buffer in a scratchpad of the memory unit;

the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and

the read operation and the write operation overlap in time.

15 . A system comprising one or more processors coupled to a memory device, the memory device to store computer program instructions that are executable by the one or more processors to perform operations comprising:

determining that a program produces an output that includes a variable length tensor;

determining a maximum size of the variable length tensor;

setting, based on the maximum size, a maximum value of a counter associated with the program;

setting a counter value of the counter to an initial value of zero;

initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;

determining that the program is operating on a first portion of the input tensor;

performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and

communicating the updated counter value to one or more consumers of the counter within the program.

16 . The system of claim 15 , wherein:

determining that the program produces the output that includes a variable length tensor is performed at a compile time.

17 . The system of claim 15 , wherein:

determining the maximum size of the variable length tensor is performed at a compile time.

18 . The system of claim 15 , the operations further comprising:

determining that the program is operating on a second portion of the input tensor;

performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and

communicating the second updated counter value to the one or more consumers of the counter within the program.

19 . The system of claim 15 , the operations further comprising:

determining that the program has completed operating on the input tensor;

determining a final size of the output; and

communicating the final size to one or more downstream consumers of the output of the program.

20 . The system of claim 15 , wherein:

the counter is located in at least one of:

a read context of a memory unit that is used when reading the first portion of the input tensor; or

a write context of the memory unit that is used when writing a portion of the output;

the write context performs a write operation to a first buffer in a scratchpad of the memory unit;

the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and

the read operation and the write operation overlap in time.

Assignments (2)

Continuity (1)

References Cited (64)