Technologies for shifted neural networks

1. A compute device for use of a shifted neural network, the compute device comprising:

a linear approximation module to determine a shift-based activation function of the shifted neural network, wherein the shift-based activation function is a piecewise linear approximation of a transcendental activation function and is defined by a plurality of line segments, and wherein a corresponding slope of each line segment is a power of two; and

a training module to train the shifted neural network based on shift-based input weights and the determined shift-based activation function,

wherein to determine the shift-based activation function comprises to: (i) compute a slope value as a base-two logarithm of a tangent line of the transcendental activation function for each function value of a plurality of function values in the range of the transcendental activation function, (ii) associate each slope value with a corresponding nearest cluster, and (iii) update a centroid associated with each cluster.

2. The compute device of claim 1 , further comprising a data handling module to identify a set of input values and a set of shift-based input weights for each neuron of the shifted neural network.

3. The compute device of claim 2 , wherein the set of input values comprises a set of predefined image patterns for recognition.

4. The compute device of claim 1 , wherein the transcendental activation function comprises a sigmoid function.

5. The compute device of claim 1 , wherein the transcendental activation function comprises a hyperbolic tangent function.

6. The compute device of claim 1 , wherein to determine the shift-based activation function based on k-means clustering further comprises to compute an offset of the tangent line relative to an origin.

7. The compute device of claim 1 , wherein to determine the shift-based activation function based on k-means clustering further comprises to compute intersections of the tangent lines in response to a determination that each centroid value is stable.

8. The compute device of claim 7 , wherein to determine the shift-based activation function comprises to generate the piecewise linear approximation according to

y

≈

{

x

⪢

σ

1

+

b

1

,

0

<

x

<

r

1

⋮

x

⪢

σ

k

+

b

k

,

r

k

-

1

<

x

<

r

k

wherein x is an input value, σ i is the slope value of the i th tangent line, b i is an offset of the i th tangent line relative to the transcendental activation function, r i is an intersection of two tangent lines, and >> is a bitwise shift operator.

9. The compute device of claim 1 , further comprising a pattern recognition module to apply the shifted neural network to an input for pattern recognition.

10. The compute device of claim 1 , wherein each corresponding slope is computed by a bitwise shift operation.

11. The compute device of claim 1 , wherein each of the input weights is stored in a four-bit memory region; and

wherein three bits of the four-bit memory region store a value associated with an amount of bitwise shift of the corresponding input weight and one bit is indicative of a sign of the weight.

12. One or more non-transitory, machine-readable storage media comprising a plurality of instructions stored thereon that, in response execution by a compute device, causes the compute device to:

determine a shift-based activation function of the shifted neural network, wherein the shift-based activation function is a piecewise linear approximation of a transcendental activation function and is defined by a plurality of line segments, and wherein a corresponding slope of each line segment is a power of two; and

train the shifted neural network based on shift-based input weights and the determined shift-based activation function,

wherein to determine the shift-based activation function comprises to (i) compute a slope value as a base-two logarithm of a tangent line of the transcendental activation function for each function value of a plurality of function values in the range of the transcendental activation function, (ii) associate each slope value with a corresponding nearest cluster, and (iii) update a centroid associated with each cluster, to determine the shift-based activation function based on k-means clustering.

13. The one or more non-transitory, machine-readable storage media of claim 12 , wherein the plurality of instructions further causes the compute device to identify a set of input values and a set of shift-based input weights for each neuron of the shifted neural network,

wherein the set of input values comprises a set of predefined image patterns for recognition.

14. The one or more non-transitory, machine-readable storage media of claim 12 , wherein the transcendental activation function comprises one of a sigmoid function or a hyperbolic tangent function.

15. The one or more non-transitory, machine-readable storage media of claim 12 , wherein to determine the shift-based activation function based on k-means clustering further comprises to:

compute an offset of the tangent line relative to an origin; and

compute intersections of the tangent lines in response to a determination that each centroid value is stable.

16. The one or more non-transitory, machine-readable storage media of claim 15 , wherein to determine the shift-based activation function comprises to generate the piecewise linear approximation according to

y

≈

{

x

⪢

σ

1

+

b

1

,

0

<

x

<

r

1

⋮

x

⪢

σ

k

+

b

k

,

r

k

-

1

<

x

<

r

k

wherein x is an input value, σ i is the slope value of the i th tangent line, b i is an offset of the i th tangent line relative to the transcendental activation function, r i is an intersection of two tangent lines, and >> is a bitwise shift operator.

17. The one or more non-transitory, machine-readable storage media of claim 12 , wherein each corresponding slope is computed by a bitwise shift operation.

18. A method for use of a shifted neural network, the method comprising:

determining, by a compute device, a shift-based activation function of the shifted neural network, wherein the shift-based activation function is a piecewise linear approximation of a transcendental activation function and is defined by a plurality of line segments, and wherein a corresponding slope of each line segment is a power of two; and

training, by the compute device, the shifted neural network based on shift-based input weights and the determined shift-based activation function,

wherein determining the shift-based activation function comprises (i) computing a slope value as a base-two logarithm of a tangent line of the transcendental activation function for each function value of a plurality of function values in the range of the transcendental activation function, (ii) associating each slope value with a corresponding nearest cluster, and (iii) updating a centroid associated with each cluster.

19. The method of claim 18 , further comprising identifying, by the compute device, a set of input values and a set of shift-based input weights for each neuron of the shifted neural network.

20. The method of claim 18 , wherein determining the shift-based activation function based on k-means clustering further comprises:

computing an offset of the tangent line relative to an origin; and

computing intersections of the tangent lines in response to a determination that each centroid value is stable.

21. The method of claim 20 , wherein determining the shift-based activation function comprises generating the piecewise linear approximation according to

y

≈

{

x

⪢

σ

1

+

b

1

,

0

<

x

<

r

1

⋮

x

⪢

σ

k

+

b

k

,

r

k

-

1

<

x

<

r

k

wherein x is an input value, σ i is the slope value of the i th tangent line, b i is an offset of the i th tangent line relative to the transcendental activation function, r i is an intersection of two tangent lines, and >> is a bitwise shift operator.