Activation functions for artificial intelligence operations

1. A memory device, comprising:

a number of memory arrays configured to store at least one of: an input, an output, a weight matrix, bias information for neural networks, or activation function information for an artificial intelligence (AI) operation performed by an AI accelerator;

a register configured to enable usage of an activation function for the AI operation;

a number of activation function registers configured to define the activation function for the AI operation performed by the AI accelerator;

a memory controller configured to select the defined activation function by programming a first bit of the number of activation function registers to a first state to execute an identity function for the AI operation and programming a second bit of the activation function registers to the first state to execute a binary step function for the AI operation, modify the defined activation function to a custom activation function, and enable performance of the custom activation function for the AI operation without a host by programming a third bit of the number of activation function registers to the first state in response to a result of a previous AI operation, wherein a different activation function was executed while performing the previous AI operation; and

the AI accelerator configured to perform the AI operation using at least one of: the input, the output, the weight matrix, the bias information for the neural networks, or the activation function information and the custom activation function.

2. The apparatus of claim 1 , wherein one or more of the number of activation function registers define a pre-defined activation function.

3. The apparatus of claim 1 , wherein one or more of the number of activation function registers define a different custom activation function.

4. The apparatus of claim 2 , wherein the pre-defined activation function is at least one of the identity function, the binary step function, a logistic function, a hyperbolic tangent function, an inverse tangent function, a softsign function, an inverse square root unit function, a rectified linear unit function, a leaky rectified linear unit function, parametric rectified linear unit function, a randomized leaky rectified linear unit function, an exponential linear unit function, or a scaled exponential linear unit function.

5. The apparatus of claim 3 , wherein the different custom activation function is based on the result of the previous AI operation.

6. The apparatus of claim 5 , wherein the previous AI operation is a debug operation.

7. The apparatus of claim 3 , wherein the different custom activation function is modified from a pre-defined activation function.

8. An apparatus, comprising:

a number of memory arrays configured to store at least one of: an input, an output, a weight matrix, bias information for neural networks, or activation function information for an artificial intelligence (AI) operation performed by an AI accelerator;

a register configured to enable usage of an activation function for the AI operation;

a memory controller configured to select the activation function by programming a first bit of an activation function register to a first state to execute an identity function for the AI operation and programming a second bit of the activation function register to the first state to execute a binary step function for the AI operation, modify the activation function to a custom activation function, and cause the apparatus to enable performance of the custom activation function for the AI operation without a host by programming a third bit in the activation function register to the first state in response to receiving a result of a previous AI operation, wherein a different activation function was executed while performing the previous AI operation; and

the AI accelerator configured to perform the AI operation using at least one of: the input, the output, the weight matrix, the bias information for the neural networks, or the activation function information and the custom activation function.

9. The apparatus of claim 8 , wherein the activation function is a pre-defined activation function.

10. The apparatus of claim 9 , wherein the third bit of the activation function register is programmed to the first state to start the custom activation function.

11. The apparatus of claim 10 , wherein the third bit of the activation function register is programmed to a second state to stop the custom activation function.

12. The apparatus of claim 10 , wherein the third bit of the activation function register is programmed to a second state to prevent the custom activation function from being executed.

13. A method, comprising:

storing at least one of: an input, an output, a weight matrix, bias information for neural networks, or activation function information for an artificial intelligence (AI) operation performed by an AI accelerator in a number of memory arrays;

enabling usage of an activation function for the AI operation using a register;

executing the activation function while performing the AI operation at the AI accelerator using at least one of: the input, the output, the weight matrix, the bias information for the neural networks, or the activation function information; and

selecting a different activation function by programming a first bit of an activation function register to a first state to execute an identity function for the AI operation and programming a second bit of the activation function register to the first state to execute a binary step function for the AI operation, modifying the different activation function to a custom activation function, and executing the custom activation function by a memory controller without a host in response to a result of the AI operation by programming a third bit in the activation function register to the first state.

14. The method of claim 13 , further comprising executing the custom activation function while performing a different AI operation.

15. The method of claim 13 , further comprising updating the different activation function in response to the result of the AI operation.

16. The method of claim 13 , wherein the different activation function is a pre-defined activation function.

17. The method of claim 13 , wherein the AI operation is a debug operation.