Neural network accelerator output ranking

1 . A system comprising:

at least one neural core;

a data memory;

a memory controller configured to access the data memory;

an electronic circuit comprising:

a plurality of comparators configured in a comparator tree, the comparator tree having a root node,

a register, and

a two-way comparator operatively connected to the root node and to the register, wherein an output of the register is coupled to a first input of the two-way comparator, wherein an output of the two-way comparator is connected to an input of the register, and wherein an output of the root node is connected to a second input of the two-way comparator, and

wherein the at least one neural core is configured to generate a first plurality of values, each value of the first plurality of values having a corresponding output neuron of an artificial neural network,

wherein the data memory is configured to receive the first plurality of values from the at least one neural core,

wherein the memory controller is configured to provide the first plurality of values from the data memory to the comparator tree,

wherein the comparator tree is configured to perform a plurality of concurrent pairwise comparisons of the first plurality of values to arrive at a greatest value of the first plurality of values,

wherein the two-way comparator is configured to:

receive, at the first input of the two-way comparator, a stored maximum value of the register,

receive, at the second input of the two-way comparator, the greatest value from the comparator tree, and

provide, at the output of the two-way comparator, a maximum value out of the stored maximum value and the greatest value, and

wherein the register is configured to store the maximum value provided from the output of the two-way comparator, and an identification of the output neuron corresponding with the maximum value, thereby replacing the stored maximum value.

2 . The system of claim 1 , wherein the memory controller is further configured to provide a second plurality of values from the data memory to the comparator tree.

3 . The system of claim 2 , wherein the memory controller is further configured to determine the second plurality of values by removing the greatest value from the first plurality of values.

4 . The system of claim 1 , wherein the comparator tree is configured to perform the plurality of concurrent pairwise comparisons in a pipelined configuration.

5 . The system of claim 1 , further comprising:

an additional comparator tree, configured to operate in parallel to the comparator tree.

6 . A system comprising:

at least one neural core;

a data memory;

a memory controller configured to access the data memory;

a plurality of k-way sorters, wherein k is greater than two;

an electronic circuit comprising:

a plurality of first k-way mergers configured in a merger tree, the merger tree having a root node,

a register, and

a second k-way merger operatively connected to the root node and to the register, wherein an output of the register is coupled to a first input of the second k-way merger, wherein an output of the second k-way merger is connected to an input of the register, and wherein an output of the root node is connected to a second input of the second k-way merger, and

wherein the at least one neural core is configured to generate a first plurality of values, each value of the first plurality of values having a corresponding output neuron of an artificial neural network,

wherein the data memory is configured to receive the first plurality of values from the at least one neural core,

wherein the memory controller is configured to provide the first plurality of values from the data memory to the plurality of k-way sorters,

wherein the plurality of k-way sorters is configured to provide sorted output of the first plurality of values to the merger tree,

wherein the merger tree is configured to perform a plurality of concurrent mergers of the first plurality of values to arrive at a set of greatest values of the first plurality of values,

wherein the second k-way merger is configured to:

receive, at the first input of the second k-way merger, stored k maximum values of the register,

receive, at the second input of the second k-way merger, a greatest k values of the set of greatest values from the merger tree, and

provide, at the output of the second k-way merger, k maximum values out of the stored k maximum values and the greatest k values of the set of greatest values, and

wherein the register is configured to store the k maximum values provided from the output of the second k-way merger, and an identification of the output neuron corresponding with each of the k maximum values, thereby replacing the stored k maximum values.

7 . The system of claim 6 , wherein: the memory controller is further configured to provide a second plurality of values from the data memory to the plurality of k-way sorters.

8 . The system of claim 7 , wherein the memory controller is further configured to determine the second plurality of values by removing the set of greatest values from the first plurality of values.

9 . The system of claim 6 , wherein the merger tree is configured to perform the plurality of concurrent mergers in a pipelined configuration.

10 . The system of claim 6 , further comprising:

an additional merger tree, configured to operate in parallel to the merger tree.

11 . A method comprising:

generating, by at least one neural core, a first plurality of values, wherein each value of the first plurality of values having a corresponding output neuron of an artificial neural network;

receiving, at a data memory, the first plurality of values from the at least one neural core;

providing the first plurality of values from the data memory to a comparator tree of an electronic circuit, wherein the electronic circuit comprises:

a plurality of comparators configured in the comparator tree, the comparator tree having a root node,

a register, and

a two-way comparator operatively connected to the root node and to the register, wherein an output of the register is coupled to a first input of the two-way comparator, wherein an output of the two-way comparator is connected to an input of the register, and wherein an output of the root node is connected to a second input of the two-way comparator;

performing, by the comparator tree, a plurality of concurrent pairwise comparisons of the first plurality of values to arrive at a greatest value of the first plurality of values;

receiving, at the first input of the two-way comparator, a stored maximum value of the register,

receiving, at the second input of the two-way comparator, the greatest value from the comparator tree;

providing, at the output of the two-way comparator, a maximum value out of the stored maximum value and the greatest value; and

storing, by the register, the maximum value provided from the output of the two-way comparator, and an identification of the output neuron corresponding with the maximum value, thereby replacing the stored maximum value.

12 . The method of claim 11 , further comprising:

providing a second plurality of values from the data memory to the comparator tree.

13 . The method of claim 12 , further comprising:

determining the second plurality of values by removing the greatest value from the first plurality of values.

14 . The method of claim 11 , wherein the comparator tree is configured to perform the plurality of concurrent pairwise comparisons in a pipelined configuration.

15 . The method of claim 11 , further comprising operating an additional comparator tree in parallel to the comparator tree.

16 . A method comprising:

generating, by at least one neural core, a first plurality of values, wherein each value of the first plurality of values having a corresponding output neuron of an artificial neural network;

receiving, at a data memory, the first plurality of values from the at least one neural core;

providing the first plurality of values from the data memory to a plurality of k-way sorters, wherein k is greater than two;

providing sorted output of the plurality of k-way sorters to a merger tree of an electronic circuit, wherein the electronic circuit comprises:

a plurality of first k-way mergers configured in the merger tree, the merger tree having a root node,

a register, and

a second k-way merger operatively connected to the root node and to the register, wherein an output of the register is coupled to a first input of the second k-way merger, wherein an output of the second k-way merger is connected to an input of the register, and wherein an output of the root node is connected to a second input of the second k-way merger;

performing, by the merger tree, a plurality of concurrent mergers of the first plurality of values to arrive at a set of greatest values of the first plurality of values;

receiving, at the first input of the second k-way merger, stored k maximum values of the register;

receiving, at the second input of the second k-way merger, greatest k values of the set of greatest values from the merger tree;

providing, at the output of the second k-way merger, k maximum values out of the stored k maximum values and the greatest k values of the set of greatest values; and

storing, by the register, the k maximum values provided from the output of the second k-way merger, and an identification of the output neuron corresponding with each of the k maximum values, thereby replacing the stored k maximum values.

17 . The method of claim 16 , further comprising:

providing a second plurality of values from the data memory to the plurality of k-way sorters.

18 . The method of claim 17 , further comprising:

determining the second plurality of values by removing the set of greatest values from the first plurality of values.

19 . The method of claim 16 , wherein the merger tree is configured to perform the plurality of concurrent mergers in a pipelined configuration.

20 . The method of claim 16 , further comprising operating an additional merger tree in parallel to the merger tree.