Locate neural network performance hot spots
Embodiments for locating performance hot spots include collecting sample data having instruction addresses, the sample data being for a neural network model and determining instructions in the instruction addresses that are performance hot spots. A listing file is used to map the instructions of the sample data that are performance hot spots to locations in a lower-level intermediate representation. A mapping file is used to map the locations of the lower-level intermediate representation that are performance hot spots to operations in one or more higher-level representations, one or more of the operations corresponding to the performance hot spots, the mapping file being generated from compiling the neural network model.
1. A computer-implemented method comprising:
collecting sample data having instruction addresses, the sample data being for a neural network model;
determining instructions in the instruction addresses that are performance hot spots;
using a listing file to map the instructions of the sample data that are performance hot spots to locations in a lower-level intermediate representation; and
using a mapping file to map the locations of the lower-level intermediate representation that are performance hot spots to operations in one or more higher-level representations, one or more of the operations corresponding to the performance hot spots, the mapping file being generated from compiling the neural network model, the mapping file comprising the lower-level intermediate representation of the neural network model and the one or more higher-level representations of the neural network model, wherein an identification of the performance hot spots is viewable in the lower-level intermediate representation through the one or more higher-level representations of the neural network model.
2. The computer-implemented method of claim 1 , further comprising using the mapping file to map the operations in the one or more higher-level representations to nodes in the neural network model.
3. The computer-implemented method of claim 1 , further comprising determining which nodes in the neural network model represent the performance hot spots based on mapping from the one or more higher-level representations which were mapped from the lower-level intermediate representation which was mapped from the sample data.
4. The computer-implemented method of claim 1 , further comprising using the mapping file to categorize the performance hot spots in different levels of compiling the neural network model, the different levels comprising the lower-level intermediate representation through the one or more higher-level representations.
5. The computer-implemented method of claim 1 , wherein the sample data further comprises information from one or more counters.
6. The computer-implemented method of claim 1 , wherein the performance hot spots comprise one or more metrics that meet or exceed one or more thresholds.
7. The computer-implemented method of claim 1 , further comprising determining one or more of the operations in the one or more higher-level representations to optimize to thereby address the performance hot spots.
8. A system comprising:
a memory having computer readable instructions; and
one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform processes comprising:
collecting sample data having instruction addresses, the sample data being for a neural network model;
determining instructions in the instruction addresses that are performance hot spots;
using a listing file to map the instructions of the sample data that are performance hot spots to locations in a lower-level intermediate representation; and
using a mapping file to map the locations of the lower-level intermediate representation to operations in one or more higher-level representations, the mapping file being generated from compiling the neural network model, the mapping file comprising the lower-level intermediate representation of the neural network model and the one or more higher-level representations of the neural network model, wherein an identification of the performance hot spots is viewable in the lower-level intermediate representation through the one or more higher-level representations of the neural network model.
9. The system of claim 8 , further comprising using the mapping file to map the operations in the one or more higher-level representations to nodes in the neural network model.
10. The system of claim 8 , further comprising determining which nodes in the neural network model represent the performance hot spots based on mapping from the one or more higher-level representations which were mapped from the lower-level intermediate representation which was mapped from the sample data.
11. The system of claim 8 , further comprising using the mapping file to categorize the performance hot spots in different levels of compiling the neural network model, the different levels comprising the lower-level intermediate representation through the one or more higher-level representations.
12. The system of claim 8 , wherein the sample data further comprises information from one or more counters.
13. The system of claim 8 , wherein the performance hot spots comprise one or more metrics that meet or exceed one or more thresholds.
14. The system of claim 8 , further comprising determining one or more of the operations in the one or more higher-level representations to optimize to thereby address the performance hot spots.
15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform processes comprising:
collecting sample data having instruction addresses, the sample data being for a neural network model;
determining instructions in the instruction addresses that are performance hot spots;
using a listing file to map the instructions of the sample data that are performance hot spots to locations in a lower-level intermediate representation; and
using a mapping file to map the locations of the lower-level intermediate representation to operations in one or more higher-level representations, the mapping file being generated from compiling the neural network model, the mapping file comprising the lower-level intermediate representation of the neural network model and the one or more higher-level representations of the neural network model, wherein an identification of the performance hot spots is viewable in the lower-level intermediate representation through the one or more higher-level representations of the neural network model.
16. The computer program product of claim 15 , further comprising using the mapping file to map the operations in the one or more higher-level representations to nodes in the neural network model.
17. The computer program product of claim 15 , further comprising determining which nodes in the neural network model represent the performance hot spots based on mapping from the one or more higher-level representations which were mapped from the lower-level intermediate representation which was mapped from the sample data.
18. The computer program product of claim 15 , further comprising using the mapping file to categorize the performance hot spots in different levels of compiling the neural network model, the different levels comprising the lower-level intermediate representation through the one or more higher-level representations.
19. The computer program product of claim 15 , wherein the sample data further comprises information from one or more counters.
20. The computer program product of claim 15 , wherein the performance hot spots comprise one or more metrics that meet or exceed one or more thresholds.