Sub-graph in frequency domain and dynamic selection of convolution implementation on a GPU
In an example, an apparatus comprises a plurality of execution units; and logic, at least partially including hardware logic, to determine a sub-graph of a network that can be executed in a frequency domain and apply computations in the sub-graph in the frequency domain. Other embodiments are also disclosed and claimed.
1. A general purpose graphic processor comprising:
a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources; and
a processing circuitry to:
determine a complete sub-graph of a convolutional neural network that can be executed in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply convolutional computations in the sub-graph in the frequency domain;
wherein the convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using the first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using the second set of execution resources.
2. The general purpose graphic processor of claim 1 , the processing circuitry to:
dynamically select a convolutional implementation based at least in part on running a short comparison for each convolution in the network; and
expose one or more embedded cast operations in a load/store instruction to support loading data for the convolutional computations in a variable integer precision.
3. The general purpose graphic processor of claim 2 , wherein:
the selection is implemented at run time.
4. The general purpose graphic processor of claim 2 , the processing circuitry to:
divide a neural network into a plurality of tiles; and
apply convolutional computations to the plurality of tiles.
5. The general purpose graphic processor of claim 4 , the processor to:
merge the results of the convolutional computations.
6. The general purpose graphics processor of claim 1 , the processing circuitry to update the predicted level of activation sparsity for the one or more layers of the convolutional neural network on-line when the convolution neural network is operated in inference mode.
7. An electronic device, comprising:
a general purpose graphics processor comprising:
a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources; and
a processing circuitry to:
determine a complete sub-graph of a convolutional neural network that can be executed in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply convolutional computations in the sub-graph in the frequency domain;
wherein the convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using the first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using the second set of execution resources; and
a memory communicatively coupled to the general purpose graphics processor.
8. The electronic device of claim 7 , the processor to:
dynamically select a convolutional implementation based at least in part on running a short comparison for each convolution in the network; and
expose one or more embedded cast operations in a load/store instruction to support loading data for the convolutional computations in a variable integer precision.
9. The electronic device of claim 8 , wherein:
the selection is implemented at run time.
10. The electronic device of claim 8 , the processor to:
divide a neural network into a plurality of tiles; and
apply convolutional computations to the plurality of tiles.
11. The electronic device of claim 10 , the processor to:
merge the results of the convolutional computations.
12. The electronic device of claim 7 , the processing circuitry to update the predicted level of activation sparsity for the one or more layers of the convolutional neural network on-line when the convolution neural network is operated in inference mode.
13. One or more non-transitory computer-readable medium comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
receive, in a general purpose graphics processor comprising a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources, data representing a convolutional neural network;
determine a complete sub-graph of a convolutional neural network that can be executed in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply convolutional computations in the sub-graph in the frequency domain;
wherein the convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using the first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using the second set of execution resources.
14. The one or more non-transitory computer-readable medium of claim 13 , comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
update the predicted level of activation sparsity for the one or more layers of the convolutional neural network on-line when the convolution neural network is operated in inference mode.
15. The one or more non-transitory computer-readable medium of claim 13 , comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
dynamically select a convolutional implementation based at least in part on running a short comparison for each convolution in the network; and
expose one or more embedded cast operations in a load/store instruction to support loading data for the convolutional computations in a variable integer precision.
16. The one or more non-transitory computer-readable medium of claim 15 , wherein:
the selection is implemented at run time.
17. The one or more non-transitory computer-readable medium of claim 15 , comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
divide a neural network into a plurality of tiles; and
apply convolutional computations to the plurality of tiles.
18. The one or more non-transitory computer-readable medium of claim 17 , comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to:
merge the results of the convolutional computations.
19. A computer-implemented method comprising:
receiving, in a general purpose graphics processor comprising a plurality of execution units comprising at least a first type of execution unit having a first set of execution resources and a second type of execution unit having a second set of execution resources, different from the first set of execution resources, data representing a convolutional neural network;
determining a complete sub-graph of a convolutional neural network that can be executed in a frequency domain;
generating a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
applying convolutional computations in the sub-graph in the frequency domain;
wherein the convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using the first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using the second set of execution resources.
20. The computer-implemented method of claim 19 , comprising:
dynamically selecting a convolutional implementation based at least in part on running a short comparison for each convolution in the network; and
exposing one or more embedded cast operations in a load/store instruction to support loading data for the convolutional computations in a variable integer precision.
21. The computer-implemented method of claim 20 , wherein:
the selecting is implemented at run time.
22. The computer-implemented method of claim 20 , comprising:
dividing a neural network into a plurality of tiles; and
applying convolutional computations to the plurality of tiles.
23. The computer-implemented method of claim 22 , comprising:
merging the results of the convolutional computations.
24. The computer-implemented method of claim 19 , comprising:
updating the predicted level of activation sparsity for the one or more layers of the convolutional neural network on-line when the convolution neural network is operated in inference mode.