Granted Patent B2

US 11,106,976 · App. 16/459,731 · Granted Aug 31, 2021

Inventor: Sylvain Flamant (Saratoga, CA)

Assignee: Wave Computing, Inc.

G06N3/08G06F16/902G06F17/16

Abstract

Claims (149)

1. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, based on a parabolic estimator function,

within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector.

2. The method of claim 1 wherein the layer is a final layer within the neural network.

3. The method of claim 1 wherein the layer that maps the first vector comprises a Softmax function.

4. The method of claim 3 wherein the layer that maps the first vector is an output layer for the neural network.

5. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network; and

the layer is based on a parabolic estimator function.

6. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function; and

the parabolic estimator function is implemented using lookup tables.

7. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables; and

the lookup tables comprise three lookup tables.

8. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables;

the lookup tables comprise three lookup tables; and

further comprising two multiplex per element of an output vector.

9. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables;

the lookup tables comprise three lookup tables;

further comprising two multiplex per element of an output vector; and

no more than three multipliers per element of an output vector are used.

10. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables;

the lookup tables comprise three lookup tables; and

the three lookup tables are each 256 elements or fewer in length.

11. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables; and

the lookup tables comprise equidistant points.

12. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables;

the lookup tables comprise equidistant points; and

values between two equidistant points with a lookup table correspond to an integer number of intervals between the two points.

13. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network,

the layer is based on a parabolic estimator function;

the parabolic estimator function is implemented using lookup tables;

the lookup tables comprise equidistant points;

values between two equidistant points with a lookup table correspond to an integer number of intervals between the two points; and

the integer number of intervals is based on a power of two.

14. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network;

the layer is based on a parabolic estimator function; and

the parabolic estimator function is implemented using one lookup table.

15. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network;

the layer is based on a parabolic estimator function; and

the parabolic estimator function is based on a moving set of three points.

16. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network;

the layer is based on a parabolic estimator function; and

the parabolic estimator function implements e Xi or k Xi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base.

17. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network;

the layer is based on a parabolic estimator function;

the parabolic estimator function implements e Xi or k Xi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base; and

k Xi is calculated using 16-bit fixed-point arithmetic.

18. A processor-implemented method for data manipulation comprising:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector;

wherein the layer that maps the first vector comprises a Softmax function,

the layer that maps the first vector is an output layer for the neural network;

the layer is based on a parabolic estimator function;

the parabolic estimator function implements e Xi or k Xi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base; and

k is equal to 2.

19. The method of claim 1 further comprising using 16-bit fixed-point arithmetic for training passes.

20. The method of claim 6 wherein lookup table entries are calculated offline.

21. A computer program product embodied in a non-transitory computer readable medium for data manipulation, the computer program product comprising code which causes one or more processors to perform operations of:

configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implementing a layer, based on a parabolic estimator function,

within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums of one using fixed point calculations; and

classifying results of the neural network based on a value of the second vector.

22. A computer system for data manipulation comprising:

a memory which stores instructions;

one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to

configure a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;

implement a layer, based on a parabolic estimator function,

within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations; and

classify results of the neural network based on a value of the second vector.

Assignments (6)

Continuity (24)

Cited By (1)

US 12,346,797