Granted Patent B2

US 12,353,006 · App. 18/590,689 · Granted Jul 8, 2025

Inventors: Philip Winterbottom (Santa Clara, CA); Martinus Bos (San Jose, CA)

Assignee: Celestial AI Inc.

G02B6/12004G02B6/12002G02B6/30G02B6/43G06F7/50G06F7/5318G06F7/5443G06N3/04G06N3/067G06N3/0675

Abstract

Claims (29)

1. A processing device comprising a plurality of neural nodes having associated sets of weights, the processing device comprising:

an electro-photonic network comprising a plurality of electronic processing elements connected by bidirectional photonic channels, each processing element comprising one or more processing engines, memory communicatively coupled to the one or more processing engines, and a message router interfacing with two or more of the bidirectional photonic channels and communicatively coupled to the memory, wherein:

each neural node is implemented by one or more of the processing elements;

the set of weights associated with each neural node is stored in the memory of the one or more processing elements implementing that neural node;

the message routers are configured, based on a predetermined flow of activations between the neural nodes, to stream a plurality of activations between the processing elements implementing the respective neural nodes during execution of implementing an artificial neural network (ANN); and

the one or more processing engines within each processing element are configured to perform, for each neural node implemented by the processing element, computations on the associated set of weights stored in the memory of the processing element and input activations to that node streamed to the processing element to generate an output activation for the node during the execution of the ANN.

2. The processing device of claim 1 , wherein each processing element implements one or more of the neural nodes.

3. The processing device of claim 1 , wherein at least one of the neural nodes is implemented by multiple of the processing elements jointly.

4. The processing device of claim 1 , wherein activations between neural nodes implemented by the same processing element are exchanged via the memory of the processing element, and wherein activations between neural nodes implemented by different processing elements are exchanged via associated ones of the bidirectional photonic channels.

5. The processing device of claim 1 , wherein the one or more processing engines comprise a dot product engine implementing a combinatorial tree of multipliers and adders configured to perform multiply-accumulate (MAC) operations.

6. The processing device of claim 5 , wherein the one or more processing engines further comprise a tensor engine configured to compute non-linear activations.

7. The processing device of claim 1 , wherein the plurality of electronic processing elements is implemented on one or more electronic integrated circuits (EICs) each comprising multiple of the electronic processing elements, and wherein the bidirectional photonic channels connecting the electronic processing elements with each EIC are implemented as optical waveguides in an associated photonic integrated circuit (PIC) stacked with the EIC.

8. The processing device of claim 7 , wherein each EIC and its associated PIC are packaged together as a system-in-package (SIP).

9. The processing device of claim 7 , wherein the plurality of electronic processing elements are implemented on multiple EICs, and wherein the bidirectional photonic channels connecting electronic processing elements on different ones of the multiple EICs are implemented with optical fiber.

10. The processing device of claim 7 , wherein the plurality of electronic processing elements are arranged in a quadrilateral grid, and wherein the bidirectional photonic channels are collectively configured to connect pairs of immediately adjacent processing elements in the grid to result in a quadrilateral mesh topology of the electro-photonic network.

11. A method comprising:

loading prior to runtime, based on an assignment between a plurality of neural nodes and a plurality of processing elements, predetermined weights associated with each neural node of the plurality of neural nodes into memory of one or more processing elements assigned to implement the neural node of the plurality of neural nodes; and

using message routers at runtime, based on the assignment and on a predetermined flow of activations between the plurality of neural nodes within an artificial neural network (ANN), to route the activations between the plurality of neural nodes implemented by the plurality of processing elements, wherein activations routed between the plurality of neural nodes implemented by a plurality of source elements and a plurality of destination elements among the plurality of processing elements are transformed, at a respective source element of the plurality of source elements and at any intermediate processing element along a path from the respective source element of the plurality of source elements to the respective destination element of the plurality of destination elements, from an electrical form to an optical form for transmission via bidirectional photonic channels, and are transformed, at the respective destination element of the plurality destination elements and any intermediate processing elements along the path, from an optical form to an electrical form.

12. The method of claim 11 , wherein the weights are loaded from memory external to an electro-photonic network into the memory of the processing elements via a PCIE interface.

13. The method of claim 11 , wherein activations between neural nodes implemented by a shared processing element are transferred electronically within the processing element.

14. The method of claim 11 , wherein the assignment between the neural nodes and the processing elements is determined at compile time.

15. The method of claim 11 , further comprising operating an electro-photonic network to:

stream activations between the processing elements in accordance with routing decisions made by the message routers; and

for each node, perform computations on the associated weights and input activations streamed to the node using one or more processing engines of the one or more processing elements assigned to implement the node.

16. The method of claim 11 , further comprising selectively activating photonic links within the bidirectional optical channels based on the flow of activations for reduced power consumption.

17. The method of claim 16 , wherein each bidirectional photonic channel comprises multiple photonic links, and wherein one of the links in each channel is kept active to serve as a master link.

18. The method of claim 11 , wherein each bidirectional photonic channel comprises multiple photonic links in each direction, the method further comprising, during initialization of an electro-photonic network, establishing logical channels over at least some of the bidirectional photonic channels, wherein establishing a logical channel over a bidirectional photonic channel comprises bonding photonic links in each direction within that bidirectional photonic channel together to form a bonding group of links.

19. The method of claim 18 , wherein photonic links that are nonfunctioning are omitted from the bonding groups.

20. The method of claim 18 , wherein a number of photonic links in the bonding group is determined based on bandwidth requirements associated with implementing the ANN.

Assignments (2)

Continuity (3)

References Cited (331)

Cited By (8)

US 12,564,047 US 12,568,809 US 12,615,087 US 12,674,948 US 12,674,949 US 12,690,438 US 12,699,240 US 12,706,680