Asynchronous schedulng and execution of data portion optimizations

1 . A system comprising:

a memory storing executable program code; and

one or more processing units to execute the executable program code to cause the system to:

determine plurality of data portion optimizations, each of the first plurality of data portion optimizations indicating a first data portion and a first optimization;

transmit the first plurality of data portion optimizations to an optimization queue;

execute each of a plurality of threads to retrieve a respective data portion optimizations from the optimization queue and execute the optimization indicated by the retrieved respective data portion optimizations on the data portion indicated by the retrieved respective data portion optimization;

while at least one of the plurality of threads is executing the indicated optimization:

determine a second plurality of data portion optimizations, each of the second plurality of data portion optimizations indicating a second data portion and a second optimization; and

transmit the second plurality of data portion optimizations to the optimization queue; and

continue to execute the plurality of threads to retrieve the second plurality of data portion optimizations from the optimization queue and execute the second optimizations on the second data portions.

2 . The system of claim 1 , wherein transmission of the first plurality of data portion optimizations to the optimization queue comprises transmission of a third optimization of a third data portion and a fourth optimization of the third data portion:

the one or more processing units to execute the executable program code to cause the system to:

determine that a priority of the third optimization is higher than a priority of the fourth optimization; and

in response to the determination that the priority of the third optimization is higher than the priority of the fourth optimization, add the third optimization to the optimization queue and do not add the fourth optimization to the optimization queue.

3 . The system of claim 2 , wherein transmission of the second plurality of data portion optimizations to the optimization queue comprises transmission of a fifth optimization of the third data portion:

the one or more processing units to execute the executable program code to cause the system to:

determine that a priority of the fifth optimization is higher than a priority of the third optimization; and

in response to the determination that the priority of the fifth optimization is higher than the priority of the third optimization, delete the third optimization from the optimization queue and add the fifth optimization to the optimization queue.

4 . The system of claim 1 , wherein transmission of the first plurality of data portion optimizations to the optimization queue comprises transmission of an optimize compression optimization of a third data portion and wherein transmission of the second plurality of data portion optimizations to the optimization queue comprises transmission of a delta merge optimization of the third data portion:

the one or more processing units to execute the executable program code to cause the system to:

determine that a priority of the optimize compression optimization of the third data portion is higher than a priority of the delta merge optimization of the third data portion based on a timestamp of the optimize compression optimization of the third data portion in the optimization queue; and

in response to the determination that the priority of the optimize compression optimization of the third data portion is higher than the priority of the delta merge optimization of the third data portion, do not add the delta merge optimization of the third data portion to the optimization queue.

5 . The system of claim 1 , wherein transmission of the first plurality of data portion optimizations to the optimization queue comprises transmission of a third optimization of a third data portion and wherein transmission of the second plurality of data portion optimizations to the optimization queue comprises transmission of a fourth optimization of the third data portion:

the one or more processing units to execute the executable program code to cause the system to:

determine that a priority of the fourth optimization is higher than a priority of the third optimization; and

in response to the determination that the priority of the fourth optimization is higher than the priority of the third optimization, delete the third optimization of the third data portion from the optimization queue and add the fourth optimization of the third data portion to the optimization queue.

6 . The system of claim 5 , wherein the fourth optimization comprises a delta merge, and wherein the third optimization comprises a compression optimization.

7 . The system of claim 1 , wherein the first plurality of data portion optimizations include a third optimization of a third data portion and wherein the second plurality of data portion optimizations include the third optimization of the third data portion:

the one or more processing units to execute the executable program code to cause the system to:

in response to the transmission of the second plurality of data portion optimizations, determine that the optimization queue includes the third optimization of the third data portion; and

in response to the determination that the optimization queue includes the third optimization of the third data portion, do not add the third optimization of the third data portion to the optimization queue.

8 . The system of claim 1 , the one or more processing units to execute the executable program code to cause the system to:

determine a number of threads based on performance-related metric values associated with the system; and

modify the number of the plurality of threads based on the determined number.

9 . A method comprising:

determining a first plurality of data portion optimizations, each of the first plurality of data portion optimizations indicating a first data portion and a first optimization;

transmitting the first plurality of data portion optimizations to an optimization queue;

executing each of a plurality of threads to retrieve a respective data portion optimization from the optimization queue and execute the optimization indicated by the retrieved respective data portion optimization on the data portion indicated by the retrieved respective data portion optimization;

while at least one of the plurality of threads is executing the indicated optimizations:

determining a second plurality of data portion optimizations, each of the second plurality of data portion optimizations indicating a second data portion and a second optimization; and

transmitting the second plurality of data portion optimizations to the optimization queue; and

continuing to execute the plurality of threads to retrieve the second plurality of data portion optimizations from the optimization queue and execute the second optimizations on the second data portions.

10 . The method of claim 9 , wherein transmitting the first plurality of data portion optimizations to the optimization queue comprises transmitting a third optimization of a third data portion and a fourth optimization of the third data portion, the method further comprising:

determining that a priority of the third optimization is higher than a priority of the fourth optimization; and

in response to determining that the priority of the third optimization is higher than the priority of the fourth optimization, adding the third optimization to the optimization queue and not adding the fourth optimization to the optimization queue.

11 . The method of claim 10 , wherein transmitting the second plurality of data portion optimizations to the optimization queue comprises transmitting a fifth optimization of the third data portion, the method further comprising:

determining that a priority of the fifth optimization is higher than a priority of the third optimization; and

in response to determining that the priority of the fifth optimization is higher than the priority of the third optimization, deleting the third optimization from the optimization queue and adding the fifth optimization to the optimization queue.

12 . The method of claim 9 , wherein transmitting the first plurality of data portion optimizations to the optimization queue comprises transmitting an optimize compression optimization of a third data portion and wherein transmitting the second plurality of data portion optimizations to the optimization queue comprises transmitting a delta merge optimization of the third data portion, the method further comprising:

determining that a priority of the optimize compression optimization of the third data portion is higher than a priority of the delta merge optimization of the third data portion based on a timestamp of the optimize compression optimization of the third data portion in the optimization queue; and

in response to determining that the priority of the optimize compression optimization of the third data portion is higher than the priority of the delta merge optimization of the first data portion, not adding the delta merge optimization of the third data portion to the optimization queue.

13 . The method of claim 9 , wherein transmitting the first plurality of data portion optimizations to the optimization queue comprises transmitting a third optimization of a third data portion and wherein transmitting the second plurality of data portion optimizations to the optimization queue comprises transmitting a fourth optimization of the third data portion, the method further comprising:

determining that a priority of the fourth optimization is higher than a priority of the third optimization; and

in response to determining that the priority of the fourth optimization is higher than the priority of the third optimization, deleting the third optimization of the third data portion from the optimization queue and adding the fourth optimization of the third data portion to the optimization queue.

14 . The method of claim 13 , wherein the fourth optimization comprises a delta merge, and wherein the third optimization comprises a compression optimization.

15 . The method of claim 9 , wherein the first plurality of data portion optimizations include a third optimization of a first third data portion and wherein the second plurality of data portion optimizations include the third optimization of the third data portion, the method further comprising:

in response to transmitting the second plurality of data portion optimizations, determining that the optimization queue includes the third data portion optimization of the third data portion; and

in response to determining that the optimization queue includes the third optimization of the third data portion, not adding the third optimization of the third data portion to the optimization queue.

16 . One or more non-transitory computer-readable media storing program code, the program code executable by a computing system to cause to the computing system to:

determine a first plurality of data portion optimizations, each of the first plurality of data portion optimizations indicating a first data portion and a first optimization;

transmit the first plurality of data portion optimizations to an optimization queue;

execute each of a plurality of threads to retrieve a respective data portion optimizations from the optimization queue and execute optimization indicated by the retrieved respective data portion optimizations on the data portion indicated by the retrieved respective data portion optimization;

while at least one of the plurality of threads is executing the indicated optimizations:

determine a second plurality of data portion optimizations, each of the second plurality of data portion optimizations indicating a second data portion and a second optimization; and

transmit the second plurality of data portion optimizations to the optimization queue; and

continue to execute the plurality of threads to retrieve the second plurality of data portion optimizations from the optimization queue and execute the second optimizations on the second data portions.

17 . The one or more non-transitory computer-readable media of claim 16 , wherein transmission of the first plurality of data portion optimizations to the optimization queue comprises transmission of a third optimization of a third data portion and a fourth optimization of the third data portion:

the program code executable by a computing system to cause to the computing system to:

determine that a priority of the third optimization is higher than a priority of the fourth optimization; and

in response to the determination that the priority of the third optimization is higher than the priority of the fourth optimization, add the third optimization to the optimization queue and do not add the fourth optimization to the optimization queue.

18 . The one or more non-transitory computer-readable media of claim 17 , wherein transmission of the second plurality of data portion optimizations to the optimization queue comprises transmission of a fifth optimization of the third data portion:

the program code executable by a computing system to cause to the computing system to:

determine that a priority of the fifth optimization is higher than a priority of the third optimization; and

in response to the determination that the priority of the fifth optimization is higher than the priority of the third optimization, delete the third optimization from the optimization queue and add the fifth optimization to the optimization queue.

19 . The one or more non-transitory computer-readable media of claim 16 , wherein transmission of the first plurality of data portion optimizations to the optimization queue comprises transmission of a third optimization of a third and wherein transmission of the second plurality of data portion optimizations to the optimization queue comprises transmission of a fourth optimization of the third data portion:

the program code executable by a computing system to cause to the computing system to:

determine that a priority of the fourth optimization is higher than a priority of the third optimization; and

in response to the determination that the priority of the fourth optimization is higher than the priority of the third optimization, delete the third optimization from the optimization queue and add the fourth optimization to the optimization queue.

20 . The one or more non-transitory computer-readable media of claim 19 , wherein the fourth optimization comprises a delta merge, and wherein the third optimization comprises a compression optimization.