ELECTRO-PHOTONIC NETWORK FOR MACHINE LEARNING
Various embodiments provide for electro-photonic networks, including a plurality of processing elements connected by bidirectional photonic channels, suited for implementing neural-network models. Weights of the model may be preloaded into memory of the processing elements based on assignments of neural nodes to processing elements implementing them, and routers of the processing elements can be configured to stream activations between the processing elements based on a predetermined flow of activations in the model.
1 . A processing device implementing an artificial neural network (ANN) comprising a plurality of neural nodes having associated sets of weights, the processing device comprising:
an electro-photonic network comprising a plurality of electronic processing elements connected by bidirectional photonic channels, each processing element comprising one or more processing engines, memory communicatively coupled to the one or more processing engines, and a message router interfacing with two or more of the bidirectional photonic channels and communicatively coupled to the memory, wherein:
each neural node of the ANN is implemented by one or more of the processing elements;
the set of weights associated with each neural node is stored in the memory of the one or more processing elements implementing that neural node;
the message routers are configured, based on a predetermined flow of activations between the neural nodes, to stream a plurality of activations between the processing elements implementing the respective neural nodes during execution of the ANN; and
the one or more processing engines within each processing element are configured to perform, for each neural node implemented by the processing element, computations on the associated set of weights stored in the memory of the processing element and input activations to that node streamed to the processing element to generate an output activation for the node during the execution of the ANN.
2 . The processing device of claim 1 , wherein each processing element implements one or more of the neural nodes.
3 . The processing device of claim 1 , wherein at least one of the neural nodes is implemented by multiple of the processing elements jointly.
4 . The processing device of claim 1 , wherein activations between neural nodes implemented by the same processing element are exchanged via the memory of the processing element, and wherein activations between neural nodes implemented by different processing elements are exchanged via associated ones of the bidirectional photonic channels.
5 . The processing device of claim 1 , wherein the one or more processing engines comprise a dot product engine implementing a combinatorial tree of multipliers and adders configured to perform multiply-accumulate (MAC) operations.
6 . The processing device of claim 5 , wherein the one or more processing engines further comprise a tensor engine configured to compute non-linear activations.
7 . The processing device of claim 1 , wherein the plurality of electronic processing elements is implemented on one or more electronic integrated circuits (EICs) each comprising multiple of the electronic processing elements, and wherein the bidirectional photonic channels connecting the electronic processing elements with each EIC are implemented as optical waveguides in an associated photonic integrated circuit (PIC) stacked with the EIC.
8 . The processing device of claim 7 , wherein each EIC and its associated PIC are packaged together as a system-in-package (SIP).
9 . The processing device of claim 7 , wherein the plurality of electronic processing elements are implemented on multiple EICs, and wherein the bidirectional photonic channels connecting electronic processing elements on different ones of the multiple EICs are implemented with optical fiber.
10 . The processing device of claim 7 , wherein the plurality of electronic processing elements are arranged in a quadrilateral grid, and wherein the bidirectional photonic channels are collectively configured to connect pairs of immediately adjacent processing elements in the grid to result in a quadrilateral mesh topology of the electro-photonic network.
11 . A method of implementing an artificial neural network (ANN) comprising a plurality of neural nodes with associated predetermined weights, the method comprising:
connecting to an electro-photonic network comprising a plurality of processing elements connected by bidirectional photonic channels, each processing element comprising one or more processing engines, memory communicatively coupled to the one or more processing engines, and a message router including two or more photonic-channel interfaces and a communicative coupling to the memory;
loading prior to runtime, based on an assignment between the neural nodes and the processing elements, the predetermined weights associated with each neural node into the memory of one or more processing elements assigned to implement the neural node; and
using the message routers at runtime, based on the assignments and on a predetermined flow of activations between the neural nodes within the ANN, to route the activations between the neural nodes implemented by the processing elements, wherein activations routed between nodes implemented by different source and destination elements among the processing elements are transformed, at the source element and at any intermediate processing element along a path from the source element to the destination element, from an electrical form to an optical form for transmission via the bidirectional photonic channels, and are transformed, at the destination element and any intermediate processing elements along the path, from an optical form to an electrical form.
12 . The method of claim 11 , wherein the weights are loaded from memory external to the electro-photonic network into the memory of the processing elements via a PCIE interface.
13 . The method of claim 11 , wherein activations between neural nodes implemented by a shared processing element are transferred electronically within the processing element.
14 . The method of claim 11 , wherein the assignment between the neural nodes and the processing elements is determined at compile time.
15 . The method of claim 11 , further comprising operating the electro-photonic network to:
stream activations between the processing elements in accordance with routing decisions made by the configured routers; and
for each node, perform computations on the associated weights and input activations streamed to the node using the one or more processing engines of the one or more processing elements assigned to implement the node.
16 . The method of claim 11 , further comprising selectively activating photonic links within the bidirectional optical channels based on the flow of activations for reduced power consumption.
17 . The method of claim 16 , wherein each bidirectional photonic channel comprises multiple photonic links, and wherein one of the links in each channel is kept active to serve as a master link.
18 . The method of claim 11 , wherein each bidirectional photonic channel comprises multiple photonic links in each direction, the method further comprising, during initialization of the electro-photonic network, establishing logical channels over at least some of the bidirectional photonic channels, wherein establishing a logical channel over a bidirectional photonic channel comprises bonding photonic links in each direction within that bidirectional photonic channel together to form a bonding group of links.
19 . The method of claim 18 , wherein photonic links that are nonfunctioning are omitted from the bonding groups.
20 . The method of claim 18 , wherein a number of photonic links in the bonding group is determined based on bandwidth requirements associated with implementing the ANN.