IP Library Granted Patent US 12,306,788
Granted Patent B2
US 12,306,788 · App. 18/383,744 · Granted May 20, 2025

Reconfigurable dataflow unit with streaming write functionality

Inventors: Manish K. Shah (Austin, TX); Ram Sivaramakrishnan (San Jose, CA); Gregory Frederick Grohoski (Bee Cave, TX); Raghu Prabhakar (San Jose, CA)
Assignee: SambaNova Systems, Inc.
G06F15/7885G06F15/8023
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,306,788
App. No.
18/383,744
Granted
May 20, 2025
Kind
B2
Abstract

A reconfigurable processing unit is disclosed, comprising a first internal network and a second internal network with different protocols, an interface to an external network with a different protocol, a first configurable unit connected to the first internal network, a second configurable unit connected to both the first internal network and the second internal network, and a third configurable unit connected to both the second internal network and the interface to the external network. The third configurable unit is configured to receive a payload from the external network and send the transaction type identifier and the source application ID to the second configurable unit over the second internal network. The second configurable unit sends information to the first configurable unit based on the transaction type identifier and the source application ID matching the local application ID retrieved from the register.

Claims (40)

1. A reconfigurable processing unit comprising:

a first internal network and a second internal network, the first internal network and the second internal network having different protocols;

an interface to an external network, the external network having a protocol different than either the first internal network or the second internal network;

a first configurable unit coupled to the first internal network;

a second configurable unit coupled to both the first internal network and the second internal network and having a register to store a local application ID; and

a third configurable unit coupled to both the second internal network and the interface to the external network, and configured to receive a payload containing a transaction type identifier, an identifier of the second configurable unit, and a source application ID from the external network, and send the transaction type identifier and the source application ID to the second configurable unit over the second internal network;

the second configurable unit configured to send information based on the transaction type identifier to the first configurable unit over the first internal network in response to the source application ID matching the local application ID retrieved from the register, and not sending information related to the payload to the first configurable unit in response to the source application ID not matching the local application ID.

2. The reconfigurable processing unit of claim 1 , the second configurable unit further configured to generate an error condition in response to the source application ID not matching the local application ID.

3. The reconfigurable processing unit of claim 1 , wherein the transaction type identifier comprises a stream write, the payload contains write data, and the information sent to the first configurable unit includes the write data.

4. The reconfigurable processing unit of claim 3 , the first configurable unit configured to store the write data in memory of the first configurable unit.

5. The reconfigurable processing unit of claim 4 , the first configurable unit configured send a token to the second configurable unit over the first internal network in response to consuming the write data stored in the memory of the first configurable unit;

the second configurable unit further configured to, in response to receiving the token from the first configurable unit, send a packet including an indication of data consumption, the local application ID, an identifier for a target reconfigurable processing unit, and an identifier for a target configurable unit of the target reconfigurable processing unit, to the third configurable unit over the second internal network; and

the third configurable unit further configured to, in response to receiving the packet from the second configurable unit, determine an address on the external network for the target reconfigurable processing unit based on the identifier of the target reconfigurable processing unit, synthesize a second payload including a transaction type ID that indicates data consumption, the local application ID, and an identifier of the reconfigurable processing unit, and send the second payload to the address on the external network.

6. The reconfigurable processing unit of claim 1 , wherein the transaction type identifier comprises an indication of data consumption by a sender of the payload, and the information sent to the first configurable unit includes a token sent over the first internal network.

7. The reconfigurable processing unit of claim 6 , the first configurable unit configured to increment a buffer size counter in response to receiving the token.

8. The reconfigurable processing unit of claim 7 , the first configurable unit further configured to determine whether the buffer size counter has a value greater than zero, and in response to the buffer size counter having a value greater than zero, decrement the buffer size counter and send data over the first internal network to the second configurable unit;

the second configurable unit further configured to, in response to receiving the data from the first configurable unit, send a packet including the data, the local application ID, an identifier for a target reconfigurable processing unit, and an identifier for a target configurable unit of the target reconfigurable processing unit to the third configurable unit over the second internal network; and

the third configurable unit further configured to, in response to receiving the packet from the second configurable unit, determine an address on the external network for the target reconfigurable processing unit based on the identifier of the target reconfigurable processing unit, synthesize a payload that includes a stream write request, the data, the local application ID, and an identifier of the reconfigurable processing unit, and send the payload to the address on the external network.

9. A method for peer-to-peer communication between a first reconfigurable processing unit (RPU) and a second RPU, the first RPU having a second configurable unit, a first configurable unit coupled to the second configurable unit by a first internal network, and a third configurable unit coupled to the second configurable unit by a second internal network, the method comprising:

storing a local application ID in a register of the second configurable unit;

receiving a payload from an external network at the third configurable unit, the payload containing a transaction type identifier, an identifier of the second configurable unit, and a source application ID;

sending the transaction type identifier and the source application ID from the third configurable unit to the second configurable unit over the second internal network; and

sending information based on the transaction type identifier to the first configurable unit over the first internal network in response to the source application ID matching the local application ID, and not sending information related to the payload to the first configurable unit in response to the source application ID not matching the local application ID.

10. The method of claim 9 , the method further comprising generating an error condition in response to the source application ID not matching the local application ID.

11. The method of claim 9 , wherein the transaction type identifier comprises a stream write, the payload contains write data, and the information sent to the first configurable unit includes the write data.

12. The method of claim 11 , further comprising storing the write data in memory of the first configurable unit.

13. The method of claim 12 , further comprising:

sending a token from the first configurable unit to the second configurable unit over the first internal network in response to consuming the write data stored in the memory of the first configurable unit;

sending a packet including an indication of data consumption, the local application ID, an identifier for the second RPU, and an identifier for a target configurable unit of the second RPU, from the second configurable unit to the third configurable unit over the second internal network in response to receiving the token from the first configurable unit at the second configurable unit;

looking up an address on the external network for the second RPU based on the identifier of the second RPU, in response to receiving the packet at the third configurable unit from the second configurable unit;

synthesizing a second payload including a transaction type ID that indicates data consumption, the local application ID, and an identifier of the first RPU; and

sending the second payload to the address on the external network.

14. The method of claim 9 , wherein the transaction type identifier comprises an indication of data consumption by a sender of the payload, and the information sent to the first configurable unit includes a token sent over the first internal network.

15. The method of claim 14 , further comprising incrementing a buffer size counter in the first configurable unit in response to receiving the token.

16. The method of claim 15 , further comprising:

determining, by the first configurable unit, whether the buffer size counter has a value greater than zero;

decrementing the buffer size counter in the first configurable unit and sending data over the first internal network from the first configurable unit to the second configurable unit in response to the buffer size counter having a value greater than zero;

sending a packet including the data, the local application ID, an identifier for a target RPU, and an identifier for a target configurable unit of the target RPU from the second configurable unit to the third configurable unit over the second internal network;

looking up an address on the external network for the second RPU at the third configurable unit based on the identifier of the target RPU; and

synthesizing a payload including a stream write request, the data, the local application ID, and an identifier of the first RPU, and send the payload to the address on the external network from third configurable unit.

Assignments (2)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Apr 18, 2025
From: SAMBANOVA SYSTEMS, INC.
To: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY, AS AGENT
Reel/Frame 070892/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2024
From: SHAH, MANISH K.; SIVARAMAKRISHNAN, RAM; GROHOSKI, GREGORY FREDERICK; PRABHAKAR, RAGHU
To: SAMBANOVA SYSTEMS, INC.
Reel/Frame 066252/0292 →
Continuity (5)
Continuation In Part 18218562 · Jul 5, 2023
Provisional Application 63405240 · Sep 9, 2022
Provisional Application 63390484 · Jul 19, 2022
Provisional Application 63389767 · Jul 15, 2022
Related Publication 20240070111A1 · Feb 29, 2024
References Cited (57)
US 8321618B1 · Keil et al. · 2012 [cited by applicant]
US 10698853B1 · Grohoski et al. · 2020 [cited by applicant]
US 10886921B1 · Koganti · 2021 [cited by examiner]
US 10908914B2 · Vorbach et al. · 2021 [cited by applicant]
US 11055141B2 · Prabhakar et al. · 2021 [cited by applicant]
US 11615038B2 · Tørudbakken · 2023 [cited by examiner]
US 20050257012A1 · Hughes · 2005 [cited by applicant]
US 20090300262A1 · Vorbach · 2009 [cited by applicant]
US 20100191911A1 · Heddes et al. · 2010 [cited by applicant]
US 20100293304A1 · Alexandron et al. · 2010 [cited by applicant]
US 20150227490A1 · Seo et al. · 2015 [cited by applicant]
US 20160055120A1 · Vorbach et al. · 2016 [cited by applicant]
US 20160188469A1 · Nagarajan et al. · 2016 [cited by applicant]
US 20180089132A1 · Atta et al. · 2018 [cited by applicant]
US 20190158396A1 · Yu et al. · 2019 [cited by applicant]
US 20190227979A1 · Ganesh · 2019 [cited by examiner]
US 20220171716A1 · Benisty et al. · 2022 [cited by applicant]
US 20230070690A1 · Mugu et al. · 2023 [cited by applicant]
US 20230244748A1 · Natarja et al. · 2023 [cited by applicant]
US 20230251839A1 · Shah et al. · 2023 [cited by applicant]
US 20230251993A1 · Shah et al. · 2023 [cited by applicant]
US 20240020261A1 · Jordan et al. · 2024 [cited by applicant]
EP 1877927B1 · 2011 [cited by applicant]
WO 2010142987A1 · 2010 [cited by applicant]
U.S. Appl. No. 18/107,613—Non-Final Rejection dated May 13, 2024, 8 pages. [cited by applicant]
U.S. Appl. No. 18/107,690—Notice of Allowance, dated Mar. 6, 2024, 9 pages. [cited by applicant]
U.S. Appl. No. 18/199,361—Non-Final Rejection dated Dec. 21, 2023, 12 pages. [cited by applicant]
U.S. Appl. No. 18/199,361—Non-Final Rejection dated Jul. 3, 2024, 17 pages. [cited by applicant]
U.S. Appl. No. 63/349,733, filed Jun. 7, 2022, Manish K. Shah. [cited by applicant]
CA 3125707—First Office Action, dated Jan. 21, 2022, 3 pages. [cited by applicant]
CA 3125707—Voluntary Amendments, dated Jan. 4, 2022, 8 pages. [cited by applicant]
EP 20702339.8—Response to Rules 161(1) and 162 Communication, filed Feb. 25, 2022, 10 pages. [cited by applicant]
EP 20702939.8—Rules 161(1) and 162 Communication, dated Aug. 18, 2021, 3 pages. [cited by applicant]
Koeplinger et al., Spatial: A Language and Compiler for Application Accelerators, PLDI '18, Jun. 18-22, 2018, Association for Computng Machinery, 16 pages. [cited by applicant]
M. Emani et al., Accelerating Scientific Applications With Sambanova Reconfigurable Dataflow Architecture, in Computing in Science & Engineering, vol. 23, No. 2, pp. 114-119, Mar. 26, 2021, [doi: 10.1109/MCSE.2021.30572… [cited by applicant]
PCT/US2020/012079—International Preliminary Report on Patentability, dated May 7, 2021, 14 pages. [cited by applicant]
PCT/US2020/012079—International Search Report and Written Opinion mailed Apr. 29, 2020, 18 pages. [cited by applicant]
PCT/US2020/012079—Second Article 34 Amendment {Response to Informal Communication by Telephone) dated 02 rebruary 2021, as filed on Apr. 2, 2021, 5 pages. [cited by applicant]
PCT/US2023/012723—International Search Report and Written Opinion, dated May 31, 2023, 13 pages. [cited by applicant]
Podobas et al, A Survey on Coarse-Grained Reconfigurable Architectures From a Performance Perspective, IEEEAccess, vol. 2020.3012084, Jul. 27, 2020, 25 pages. [cited by applicant]
Prabhakar et al., Plasticine: A Reconfigurable Architecture for Parallel Patterns, ISCA, Jun. 24-28, 2017, 14 pages. [cited by applicant]
TW 108148376—Notice of Allowance dated Oct. 23, 2020, 5 pages. [cited by applicant]
TW 108148376—Request for Exam and Voluntary Amendment filed Jun. 30, 2020, 17 pages. [cited by applicant]
TW 110101760—First Office Action dated Mar. 29, 2022, 12 pages. [cited by applicant]
TW110101760—Notice of Allowance, dated Sep. 21, 2022, 2 pages. [cited by applicant]
U.S. Appl. No. 16/239,252—Notice of Allowance dated Feb. 12, 2020, 10 pages. [cited by applicant]
U.S. Appl. No. 16/239,252—Notice of Allowance dated May 14, 2020, 15 pages. [cited by applicant]
U.S. Appl. No. 16/239,252—Office Action dated Aug. 7, 2019, 8 pages. [cited by applicant]
U.S. Appl. No. 16/239,252—Response to Final Office Action dated Jan. 8, 2020 filed Jan. 24, 2020, 14 pages. [cited by applicant]
U.S. Appl. No. 16/239,252—Response to Office Action dated Aug. 7, 2019, filed Sep. 26, 2019, 6 pages. [cited by applicant]
U.S. Appl. No. 16/239,252 Final Office Action, dated Jan. 8, 2020, 13 pages. [cited by applicant]
U.S. Appl. No. 16/862,445—Notice of Allowance, dated Sep. 17, 2021, 15 pages. [cited by applicant]
U.S. Appl. No. 16/862,445—Response to Office Action dated Mar. 18, 2021, filed Jun. 9, 2021, 12 pages. [cited by applicant]
Ming et al., A Reconfigurable Soc for Block Ciphers with a Programmable Dataflow Structure, dated Apr. 18-20, 2011, Third International Conference on Communications and Mobile Computing, 4 pages. [cited by applicant]
Prabhakar et al., SambaNova SN10 RDU: A 7nm Dataflow Architecture to Accelerate Software 2.0, dated Feb. 20-26, 2022, IEEE International Solid-State Circuits Conference, 3 pages. [cited by applicant]
U.S. Appl. No. 18/383,718—Notice of Allowance, dated Jul. 15, 2024, 12 pages. [cited by applicant]
Yan Ming, Yang Ziyu, Li Sikun, “A Reconfigurable Soc for Block Ciphers with a Programmable Dataflow Structure,” 2011 ThirdInternational Conference on Communications and Mobile Computing (Year: 2011) 4 pages. [cited by applicant]