Optimizing resource allocation for projects executing in a cloud-based environment
Embodiments are directed towards a system and method for a cloud-based front end that may abstract and enable access to the underlying cloud-hosted elements and objects that may be part of a multi-tenant application, such as a search application. Search objects may be employed to access indexed objects. An amount of indexed data accessible to a user may be based on an index storage limit selected by the user, such that data that exceeds the index storage limit may continue to be indexed. Also, one or more projects can be elastically scaled for a user to provide resources that may meet the specific needs of each project.
1. A method implemented using a computing device, comprising:
identifying a resource allocation associated with a project, wherein the resource allocation identifies computing resources assigned to the project, and wherein the project uses the computing resources to receive data and to store the data in a dynamically scalable index store;
determining, based on one or more rules associated with the project, that the resource allocation is suboptimal;
determining, based on the one or more rules, an optimal resource allocation for the project; and
adjusting the computing resources based on the optimal resource allocation.
2. The method of claim 1 , wherein the dynamically scalable index store is accessible by an indexer server, and the indexer server is associated with a virtual machine.
3. The method of claim 1 , further comprising:
determining that a data volume limit assigned to the project is exceeded; and
causing a portion of data that exceeds the data volume limit to be stored in an overflow index store.
4. The method of claim 1 , wherein the data stored in the dynamically scalable index store is available to a user of the project, and wherein data stored in an overflow index store is unavailable to the user of the project.
5. The method of claim 1 , further comprising:
based on determining that a data volume limit assigned to the project has been increased, copying data stored in an overflow index store to the dynamically scalable index store.
6. The method of claim 1 , further comprising:
removing data from an overflow index store based on determining that the portion of data has been stored in the overflow index store for more than a predetermined amount of time.
7. The method of claim 1 , further comprising:
removing data from an overflow index store based on determining that an amount of data stored in the overflow index store has exceeded a maximum overflow index store volume.
8. The method of claim 1 , wherein the one or more rules specify an optimal speed for indexing data corresponding to the project.
9. The method of claim 1 , wherein the one or more rules include a rule-based trigger that is triggered when a speed for indexing the data corresponding to the project falls below a threshold.
10. The method of claim 1 , wherein the one or more rules include a scheduled trigger for adjusting the computing resources assigned to the project.
11. The method of claim 1 , wherein determining the optimal resource allocation comprises determining a change in a first subset of the computing resources associated with a first processing dimension based on a performance characteristic of the project.
12. The method of claim 1 , wherein determining the optimal resource allocation comprises:
determining a first change in a first subset of the computing resources associated with a first processing dimension based on a performance characteristic of the project; and
determining, based on the first change, a second change in a second subset of the computing resources associated with a second processing dimension.
13. The method of claim 1 , wherein adjusting the computing resources comprises placing a request to adjust the computing resources in a request queue until the request can be fulfilled.
14. The method of claim 1 , further comprising:
determining an impact of the optimal resource allocation on an overall performance of an indexing system that indexes data corresponding to a plurality of projects that includes the project; and
delaying a request to adjust the computing resources based on the impact.
15. The method of claim 1 , further comprising causing display of at least one user interface control for adjusting at least one parameter related to the project, wherein determining the optimal resource allocation is performed in response to input from the at least one user interface control.
16. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to:
identify a resource allocation associated with a project, wherein the resource allocation identifies computing resources assigned to the project, and wherein the project uses the computing resources to receive data and to store the data in a dynamically scalable index store;
determine, based on one or more rules associated with the project, that the resource allocation is suboptimal;
determine, based on the one or more rules, an optimal resource allocation for the project; and
adjust the computing resources based on the optimal resource allocation.
17. The one or more non-transitory computer-readable media of claim 16 , wherein the dynamically scalable index store is accessible by an indexer server, and the indexer server is associated with a virtual machine.
18. The one or more non-transitory computer-readable media of claim 16 , wherein the data stored in the dynamically scalable index store is available to a user of the project, and wherein data stored in an overflow index store is unavailable to the user of the project.
19. The one or more non-transitory computer-readable media of claim 16 , further comprising:
based on determining that a data volume limit assigned to the project has been increased, copying data stored in an overflow index store to the dynamically scalable index store.
20. The one or more non-transitory computer-readable media of claim 16 , wherein the one or more rules specify an optimal speed for indexing data corresponding to the project.
21. The one or more non-transitory computer-readable media of claim 16 , wherein the one or more rules include a rule-based trigger that is triggered when a speed for indexing the data corresponding to the project falls below a threshold.
22. The one or more non-transitory computer-readable media of claim 16 , wherein determining the optimal resource allocation comprises determining a change in a first subset of the computing resources associated with a first processing dimension based on a performance characteristic of the project.
23. The one or more non-transitory computer-readable media of claim 16 , wherein determining the optimal resource allocation comprises:
determining a first change in a first subset of the computing resources associated with a first processing dimension based on a performance characteristic of the project; and
determining, based on the first change, a second change in a second subset of the computing resources associated with a second processing dimension.
24. A computer system comprising:
a memory storing instructions; and
a processor for executing the instructions to perform the steps of:
identifying a resource allocation associated with a project, wherein the resource allocation identifies computing resources assigned to the project, and wherein the project uses the computing resources to receive data and to store the data in a dynamically scalable index store;
determining, based on one or more rules associated with the project, that the resource allocation is suboptimal;
determining, based on the one or more rules, an optimal resource allocation for the project; and
adjusting the computing resources based on the optimal resource allocation.
25. The computer system of claim 24 , wherein the dynamically scalable index store is accessible by an indexer server, and the indexer server is associated with a virtual machine.
26. The computer system of claim 24 , wherein the data stored in the dynamically scalable index store is available to a user of the project, and wherein data stored in an overflow index store is unavailable to the user of the project.
27. The computer system of claim 24 , further comprising:
based on determining that a data volume limit assigned to the project has been increased, copying data stored in an overflow index store to the dynamically scalable index store.
28. The computer system of claim 24 , wherein the one or more rules specify an optimal speed for indexing data corresponding to the project.
29. The computer system of claim 24 , wherein the one or more rules include a rule-based trigger that is triggered when a speed for indexing the data corresponding to the project falls below a threshold.
30. The computer system of claim 24 , wherein determining the optimal resource allocation comprises determining a change in a first subset of the computing resources associated with a first processing dimension based on a performance characteristic of the project.