Dynamic variable bit width neural processor
Embodiments relate to an electronic device that includes a neural processor having multiple neural engine circuits that operate in multiple modes of different bit width. A neural engine circuit may include a first multiply circuit and a second multiply circuit. The first and second multiply circuits may be combined to work as a part of a combined computation circuit. In a first mode, the first multiply circuit generates first output data of a first bit width by multiplying first input data with a first kernel coefficient. The second multiply circuit generates second output data of the first bit width by multiplying second input data with a second kernel coefficient. In a second mode, the combined computation circuit generates third output data of a second bit width by multiplying third input data with a third kernel coefficient.
1 . A device, comprising:
a first processor circuit configured to process data in a first mode based on a first bit width and a second mode based on a second bit width wider than the first bit width; and
a second processor circuit configured to provide commands to the first processor circuit, the commands comprising a first command specifying that the first processor circuit is to be operated in the first mode and a second command specifying that the first processor circuit is to be operated in the second mode,
wherein the first processor circuit comprises:
a first multiply circuit configured to perform a first multiplication operation based on input data and a first kernel coefficient to generate first output data of the first bit width in response to receiving the first command; and
a second multiply circuit configured to perform a second multiplication operation based on the input data and a second kernel coefficient to generate second output data of the second bit width in response to receiving the second command.
2 . The device of claim 1 , wherein, in response to receiving the first command, the first processor circuit is further configured to perform a convolution operation at a first layer of a machine learning model and, in response to receiving the second command, the first processor circuit is configured to perform the convolution operation at a second layer of the machine learning model, the first layer preceding the second layer in the machine learning model.
3 . The device of claim 1 , wherein, in response to receiving the first command, the first processor circuit is configured to perform coarse object detection and, in response to receiving the second command, the first processor circuit is configured to perform refined object detection.
4 . The device of claim 1 , wherein, in response to using a first power source, the second processor circuit is configured to provide the first command and, in response to using a second power source different from the first power source, the second processor circuit is configured to provide the second command.
5 . The device of claim 1 , wherein in response to the first processor circuit receiving the second command, the first multiply circuit and the second multiply circuit are configured to operate as a combined computation circuit to generate third output data of the second bit width.
6 . The device of claim 1 , wherein the first processor circuit is configured to receive the second command based on a value of a loss function during training of a machine learning model.
7 . The device of claim 1 , wherein the first processor circuit is configured to receive the second command based on a prediction boundary between two classifications of a machine learning model.
8 . A method, comprising:
processing, by a first processor circuit, data in a first mode based on a first bit width and a second mode based on a second bit width wider than the first bit width; and
providing, by a second processor circuit, commands to the first processor circuit, the commands comprising a first command specifying that the first processor circuit is to be operated in the first mode and a second command specifying that the first processor circuit is to be operated in the second mode,
wherein processing the data comprises:
performing, by a first multiply circuit of the first processor circuit, a first multiplication operation based on input data and a first kernel coefficient to generate first output data of the first bit width in response to receiving the first command; and
performing, by a second multiply circuit of the first processor circuit, a second multiplication operation based on the input data and a second kernel coefficient to generate second output data of the second bit width in response to receiving the second command.
9 . The method of claim 8 , further comprising:
performing a convolution operation at a first layer of a machine learning model in the first mode in response to the first processor circuit receiving the first command; and
performing the convolution operation at a second layer of the machine learning model in the second mode in response to the first processor circuit receiving the second command, the first layer preceding the second layer in the machine learning model.
10 . The method of claim 8 , further comprising:
performing coarse object detection in response to receiving the first command; and
performing refined object detection in response to receiving the second command.
11 . The method of claim 8 , further comprising:
providing, by the second processor circuit, the first command in response to a device, including the first processing circuit and the second processing circuit, using a first power source; and
providing, by the second processor circuit, the second command in response to the device using a second power source different from the first power source.
12 . The method of claim 8 , further comprising operating the first multiply circuit and the second multiply circuit as a combined computation circuit to generate third output data of the second bit width in response to the first processor circuit receiving the second command.
13 . The method of claim 8 , wherein providing the commands comprises providing the second command to the first processor circuit based on a value of a loss function during training of a machine learning model.
14 . The method of claim 8 , wherein providing the commands comprises providing the second command to the first processor circuit based on a prediction boundary between two classifications of a machine learning model.
15 . A system, comprising:
a data processor circuit configured to store input data; and
a neural engine circuit configured to receive commands comprising a first command specifying that the neural engine circuit is to be operated in a first mode and a second command specifying that the neural engine circuit is to be operated in a second mode, wherein, in the first mode, the neural engine circuit is configured to process the input data based on a first bit width, and, in the second mode, the neural engine circuit is configured to process the input data based on a second bit width that is wider than the first bit width, the neural engine circuit comprising:
a first multiply circuit configured to perform a first multiplication operation based on the input data and a first kernel coefficient to generate first output data of the first bit width in response to receiving the first command; and
a second multiply circuit configured to perform a second multiplication operation based on the input data and a second kernel coefficient to generate second output data of the second bit width in response to receiving the second command.
16 . The system of claim 15 , wherein, in response to receiving the first command, the neural engine circuit is further configured to perform a convolution operation at a first layer of a machine learning model in the first mode and, in response to receiving the second command, the neural engine circuit is to perform the convolution operation at a second layer of the machine learning model in the second mode, the first layer preceding the second layer in the machine learning model.
17 . The system of claim 15 , wherein, in response to receiving the first command, the neural engine circuit is configured to perform coarse object detection and, in response to receiving the second command, the neural engine circuit is configured to perform refined object detection.
18 . The system of claim 15 , wherein the second command causes the first multiply circuit and the second multiply circuit to operate as a combined computation circuit to generate third output data of the second bit width.
19 . The system of claim 15 , wherein the second command is received by the neural engine circuit based on a value of a loss function during training of a machine learning model.
20 . The system of claim 15 , wherein the second command is received by the neural engine circuit based on a prediction boundary between two classifications of a machine learning model.