Multistep automated scaling for cluster containers

1. A system for managing a cluster computing system, comprising:

a storage system configured to:

store a resource usage history for a set of tasks running on a computer cluster comprising a plurality of worker systems;

a processor configured to:

determine required resource sizes for a task of the set of tasks based at least in part on the resource usage history for the task;

resize resources allocated to the task to the required resource sizes;

determine a resource of the resources having a smallest resource availability factor;

arrange tasks of the set of tasks on the plurality of worker systems to reduce a number of worker systems running tasks, comprising to:

select an unchecked task of the set of tasks in an order of high to low resource usage for the resource, wherein a first worker system of the plurality of worker systems runs the unchecked task;

select a second worker system of the plurality of worker systems in an order of high to low loading for the resource;

move the unchecked task to the second worker system in response to a determination that the second worker system has available capacity for the required resource size of the resource for the unchecked task, wherein the first worker system is different from the second worker system; and

deallocate worker systems no longer running tasks.

2. The system of claim 1 , wherein the required resource size for the task is based at least in part on resource usage history over a period of a day, a week, a month, or a year.

3. The system of claim 1 , wherein the required resource size for the task is based at least in part on a maximum observed resource usage.

4. The system of claim 1 , wherein the resource of the resources allocated to the task comprises one of a memory utilization, storage system utilization, a processor utilization, a graphics processing unit utilization, memory bandwidth utilization, available port usage, or a network bandwidth utilization.

5. The system of claim 1 , wherein resizing the resources allocated to the task comprises reducing an amount of resources allocated to the task.

6. The system of claim 1 , wherein resizing the resources allocated to the task comprises increasing an amount of resources allocated to the task.

7. The system of claim 6 , wherein increasing the amount of resources allocated to the task comprises determining whether resources are available to increase the amount of resources allocated to the task to the required resource size.

8. The system of claim 7 , wherein, in response to determining that the resources are not available, the amount of resources allocated to the task are increased to a maximum available size.

9. The system of claim 7 , wherein, in response to determining that the resources are not available, an indication of the required resource size for the task is stored.

10. The system of claim 1 , wherein resizing the resources allocated to the task to the required resource size comprises stopping the task and restarting the task with the resource allocation set to the required resource size.

11. The system of claim 1 , wherein resizing the resources allocated to the task to the required resource size comprises resizing the resources allocated to the task without stopping the task.

12. The system of claim 1 , wherein the processor is further configured to determine a single required resource size for all tasks of a task group, wherein the task group comprises a subset of the set of tasks comprising multiple instances of a same task or a group of related tasks.

13. The system of claim 1 , wherein the required resource size comprises a projected required resource size.

14. The system of claim 1 , wherein deallocating cluster machines no longer running tasks comprises shutting down one or more physical worker systems no longer running tasks.

15. The system of claim 1 , wherein deallocating cluster machines no longer running tasks comprises releasing one or more virtual worker systems no longer running tasks.

16. The system of claim 1 , wherein the processor is further configured to receive a performance target for the task of the set of tasks.

17. The system of claim 16 , wherein the required resource size for the task is based at least in part on the performance target.

18. The system of claim 1 , wherein the required resource size is determined for each task of the set of tasks.

19. The system of claim 18 , wherein resources allocated to each task of the set of tasks are resized according to the required resource size.

20. The system of claim 1 , wherein the loading of the second worker system is based at least in part on the available capacity of the second worker system.

21. The system of claim 1 , wherein the resource availability factor is a percentage of the available resources of the plurality of worker systems not allocated to the set of tasks.

22. A method for managing a cluster computing system, comprising:

storing a resource usage history for a set of tasks running on a computer cluster comprising a plurality of worker systems;

determining, using a processor, required resource sizes for a task of the set of tasks based at least in part on the resource usage history for the task;

resizing resources allocated to the task to the required resource sizes;

determining a resource of the resources having a smallest resource availability factor;

arranging the set of tasks on the plurality of worker systems to reduce a number of worker systems running tasks, comprising:

selecting an unchecked task of the set of tasks in an order of high to low resource usage for the resource, wherein a first worker system of the plurality of worker systems runs the unchecked task;

selecting a second worker system of the plurality of worker systems in an order of high to low loading for the resource;

moving the unchecked task to the second worker system in response to a determination that the second worker system has available capacity for the required resource size of the resource for the unchecked task, wherein the first worker system is different from the second worker system; and

deallocating worker systems no longer running tasks.

23. A computer program product for managing a cluster computing system, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

storing a resource usage history for a set of tasks running on a computer cluster comprising a plurality of worker systems;

determining required resource sizes for a task of the set of tasks based at least in part on the resource usage history for the task;

resizing resources allocated to the task to the required resource sizes;

determining a resource of the resources having a smallest resource availability factor;

arranging the set of tasks on the plurality of worker systems to reduce a number of worker systems running tasks, comprising:

selecting an unchecked task of the set of tasks in an order of high to low resource usage for the resource, wherein a first worker system of the plurality of worker systems runs the unchecked task;

selecting a second worker system of the plurality of worker systems in an order of high to low loading for the resource;

moving the unchecked task to the second worker system in response to a determination that the second worker system has available capacity for the required resource size of the resource for the unchecked task, wherein the first worker system is different from the second worker system; and

deallocating worker systems no longer running tasks.