Partitioning code bases for parallel execution of code analysis
A partitioning technique is applied to divide input code into different portions. Different partitioning techniques can be applied in order to optimize the portioning of the code to account for various features of the code, such as code dependencies. Once partitioned, the code analysis tasks execute in parallel on the code portions. In this way, improved code analysis performance is obtained. Moreover, the addition of new code analysis tasks may not impact overall analysis performance as the partitioning can help to offset added or unknown analysis latency of new code analysis tasks.
1. A system, comprising:
at least one processor; and
a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to implement a code analysis system:
receive, via an interface of the code analysis system, a code base for analysis;
select a partitioning technique to apply to the code base based, at least in part, on one or more selections of a plurality of different code analysis tasks to respectively perform on each portion of the code base, wherein the partitioning technique determines how to divide the code base into portions;
apply the selected partitioning technique to the code base to divide the code base into a plurality of code portions;
direct parallel execution of the plurality of different code analysis tasks on the plurality of code portions to determine respective results for the different code analysis tasks based, at least in part, on aggregating portion-specific results for the plurality of code portions for individual ones of the different code analysis tasks; and
send, via the interface, the respective results of the different code analysis tasks.
2. The system of claim 1 , wherein the plurality of different code analysis tasks are selected as part of a request to perform the analysis on the code base received at the code analysis system.
3. The system of claim 1 , wherein the code analysis system is further configured to update performance of the partitioning technique upon a different code base based, at least in part, performance of the parallel execution of the plurality of different code analysis tasks on the plurality of code portions.
4. The system of claim 1 , wherein the code analysis system is implemented as part of a provider network, wherein the code analysis system is configured to receive the request to perform the analysis on the code base from another service of the provider network executing a deployment pipeline.
5. A method, comprising:
performing, by one or more computing devices implementing a code analysis system:
obtaining a code base for analysis;
selecting a partitioning technique to apply to the code base based, at least in part, on one or more selections of a plurality of different code analysis tasks to respectively perform on each portion of the code base, wherein the partitioning technique determines how to divide the code base into portions;
applying the partitioning technique to the code base to divide the code base into a plurality of code portions;
initiating parallel execution of the plurality of different code analysis tasks on the plurality of code portions to determine respective results for the different code analysis tasks based, at least in part, on aggregating portion-specific results for the plurality of code portions for individual ones of the different code analysis tasks; and
providing the respective results of the different code analysis tasks.
6. The method of claim 5 , wherein the plurality of different code analysis tasks are selected as part of a request to perform the analysis on the code base received at the code analysis system.
7. The method of claim 5 , wherein applying the partitioning technique comprises applying a split-merge technique.
8. The method of claim 5 , wherein applying the partitioning technique comprises applying a size limiting technique.
9. The method of claim 5 , further comprising updating performance of the partitioning technique upon a different code base based, at least in part, performance of the parallel execution of the plurality of different code analysis tasks on the plurality of code portions.
10. The method of claim 5 , wherein the plurality of different code analysis tasks are performed within a specified time limit specified as part of a received request to perform the analysis on the code base at the code analysis system.
11. The method of claim 5 , wherein the plurality of code portions comprise two or more overlapping code portions.
12. The method of claim 5 , wherein the partitioning technique is applied based on a partitioning configuration received as part of a request to perform the analysis on the code base at the code analysis system.
13. The method of claim 5 , further comprising receiving a request from a code editor application to perform the analysis on the code base.
14. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement:
receiving a code base for analysis;
selecting a partitioning technique to apply to the code base based, at least in part, on one or more selections of a plurality of different code analysis tasks to respectively perform on each portion of the code base, wherein the partitioning technique determines how to divide the code base into portions;
applying the partitioning technique to the code base to divide the code base into a plurality of code portions;
causing parallel execution of the plurality of different code analysis tasks on the plurality of code portions to determine respective results for the different code analysis tasks based, at least in part, on aggregating portion-specific results for the plurality of code portions for individual ones of the different code analysis tasks; and
providing the respective results of the different code analysis tasks.
15. The one or more non-transitory, computer-readable storage media of claim 14 , wherein the plurality of different code analysis tasks are selected as part of a received request to perform the analysis on the code base.
16. The one or more non-transitory, computer-readable storage media of claim 14 , wherein the plurality of different code analysis tasks are performed within a specified time limit specified as part of a received request to perform the analysis on the code base.
17. The one or more non-transitory, computer-readable storage media of claim 14 , wherein applying the partitioning technique comprises applying a split-merge technique.
18. The one or more non-transitory, computer-readable storage media of claim 14 , wherein the plurality of code portions comprise two or more overlapping code portions.
19. The one or more non-transitory, computer-readable storage media of claim 14 , wherein the plurality of code portions comprise two or more non-overlapping code portions.
20. The one or more non-transitory, computer-readable storage media of claim 14 , wherein the one or more computing devices are implemented as part of a service of a provider network, wherein the one or more non-transitory, computer-readable storage media store further program instructions that cause the one or more computing devices to further implement receiving the request to perform the analysis on the code base from another service of the provider network executing a deployment pipeline.