Networks on chip (NOC) for many-core neural network accelerator
This application describes a network-on-chip system on a hardware accelerator for accelerating neural network computations. An example NoC system in the NN accelerator may include interconnected routers with routing control circuits and cores respectively coupled to the routers. The cores are arranged into a matrix. Each row of cores are connected with a first uni-directional ring-shape data link and every two adjacent data links are in opposite directions. Each column of cores are connected with a second uni-directional ring-shape data link and every two adjacent data links are in opposite directions. In a given router of the plurality of routers, the routing control circuit is configured to: receive a data package; convert physical addresses of the given router and the target router into logical addresses; determine a routing port of the given router based on the logical addresses; and output the data package through the routing port.
1 . A many-core neural network (NN) accelerator, comprising:
a plurality of routers, each router comprising a routing control circuit, and
a plurality of cores respectively coupled to the plurality of routers to form a Network-on-Chip (NoC);
wherein:
for a given core of the plurality of cores, the routing control circuit of the router coupled to the given core is configured to:
detect defective cores within the NoC;
classify the defective cores as defective-with-bypass or defective-without-bypass; and
dynamically reconfigure the NoC by adjusting routing paths to disable a NoC routing port of the router coupled to a core classified as defective-without-bypass and to keep the defective cores classified as defective-with-bypass as passive conduits for data routing.
2 . The NN accelerator of claim 1 , wherein to classify the defective cores, the routing control circuit is configured to:
classify a core as defective-with-bypass in response to an internal buffer of the core being inaccessible.
3 . The NN accelerator of claim 1 , wherein the NoC comprises:
an outer-layer NoC configured for data transfer between a host device and the cores;
a middle-layer NoC configured for data transfer among the cores; and
an inner-layer NoC within each core configured for data transfer among processing elements within the core.
4 . The NN accelerator of claim 1 , wherein to dynamically reconfigure the NoC, the routing control circuit is configured to adjust physical to logical address mappings.
5 . The NN accelerator of claim 1 , wherein to detect the defective cores, the routing control circuit is configured to monitor routing timeouts or missed heartbeats.
6 . The NN accelerator of claim 1 , wherein to dynamically reconfigure the NoC, the routing control circuit is configured to downgrade the NoC to a smaller configuration with a smaller number of cores and routers.
7 . The NN accelerator of claim 1 , wherein the plurality of cores are arranged, in a logic view, as a matrix, each row of the matrix comprising a same number of cores, and the router coupled to the given core comprises three ports, including a first port connected the given core to a horizontal ring formed by a row of cores, a second port connected the given core to a vertical ring formed by a column of cores, and a third port connected to an internal buffer of the given core.
8 . The NN accelerator of claim 7 , wherein to classify the defective cores, the routing control circuit is configured to:
classify the given core as defective-with-bypass in response to the third port being defective but the first port or the second port not being defective.
9 . The NN accelerator of claim 7 , wherein to classify the defective cores, the routing control circuit is configured to:
classify the given core as defective-without-bypass in response to the first port or the second port being defective but the third port not being defective.
10 . The NN accelerator of claim 1 , wherein the routing control circuit of the router is further configured to manage routing logics for data read from an internal buffer of the coupled core and data received from another router.
11 . The NN accelerator of claim 10 , wherein to manage routing logics for data read from an internal buffer of the corresponding core and data received from another router, the routing control circuit of a router in the plurality of routers is configured to:
convert a physical address of the router to a first logical address;
in response to the router receiving a data package for routing to a target router, convert a physical address of the target router into a second logical address;
determine a routing port of the router based on the first and second logical addresses; and
route the data package through the routing port of the router.
12 . A method, comprising:
detecting defective cores within a Network-on-Chip (NoC) formed by a plurality of cores, wherein the plurality of cores are respectively coupled to a plurality of routers, each router comprising a routing control circuit;
classifying the defective cores as defective-with-bypass or defective-without-bypass; and
dynamically reconfiguring the NoC by adjusting routing paths to disable a NoC routing port of the router coupled to a core classified as defective-without-bypass and to keep the defective cores classified as defective-with-bypass as passive conduits for data routing.
13 . The method of claim 12 , wherein the plurality of cores are arranged, in a logic view, as a matrix, each row of the matrix comprising a same number of cores, and
the router coupled to a given core comprises three ports, including a first port connected the given core to a horizontal ring formed by a row of cores, a second port connected the given core to a vertical ring formed by a column of cores, and a third port connected to an internal buffer of the given core.
14 . The method of claim 13 , wherein the classifying the defective cores as defective-with-bypass or defective-without-bypass comprises:
classifying the given core as defective-with-bypass in response to the third port being defective but the first port or the second port not being defective.
15 . The method of claim 13 , wherein the classifying the defective cores as defective-with-bypass or defective-without-bypass comprises:
classifying the given core as defective-without-bypass in response to the first port or the second port being defective but the third port not being defective.
16 . The method of claim 12 , wherein the dynamically reconfiguring the NoC comprises:
downgrading the NoC to a smaller configuration with a smaller number of cores and routers.
17 . The method of claim 12 , wherein the detecting defective cores comprises:
monitoring routing timeouts or missed heartbeats of the plurality of routers.
18 . The method of claim 12 , wherein the NoC comprises:
an outer-layer NoC configured for data transfer between a host device and the cores;
a middle-layer NoC configured for data transfer among the cores; and
an inner-layer NoC within each core configured for data transfer among processing elements within the core.