Cache system with multiple cache unit states

1. A method comprising:

determining, by a processing device, that a hit ratio is below a first hit ratio threshold associated with a first cache unit and above a second hit ratio threshold associated with a second cache unit, wherein the first hit ratio threshold is different from the second hit ratio threshold; and

responsive to determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit, loading a dataset into the first cache unit rather than the second cache unit, wherein the first cache unit and the second cache unit are available to load the dataset.

2. The method of claim 1 , further comprising:

loading a first dataset into the first cache unit and the second cache unit prior to determining that the hit ratio is below the first hit ratio threshold and above the second hit ratio threshold, wherein the dataset loaded into the first cache unit rather than the second cache unit is a second dataset.

3. The method of claim 1 , further comprising:

determining that the hit ratio is below the second hit ratio threshold associated with the second cache unit; and

responsive to determining that the hit ratio is below the second hit ratio threshold associated with the second cache unit, loading a third dataset into the second cache unit.

4. The method of claim 1 , further comprising:

determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and equal to the second hit ratio threshold associated with the second cache unit; and

responsive to determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and equal to the second hit ratio threshold associated with the second cache unit, loading the dataset into the first cache unit.

5. The method of claim 2 , further comprising:

receiving an alteration to the first dataset; and

in response to receiving the alteration, loading an altered first dataset into a third cache unit.

6. The method of claim 1 , further comprising:

receiving a request to access a data store; and selecting the first cache unit to service the request.

7. The method of claim 1 , wherein determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit comprises:

identifying a first number indicative of a number of data requests received by the first cache unit and second cache unit within a window, each data request requesting a data unit;

identifying a second number indicative of a number of the data requests requesting a data unit in a first dataset within the window;

identifying the hit ratio of the first cache unit and second cache unit by dividing the second number by the first number; and

comparing the hit ratio to the first hit ratio threshold and the second hit ratio threshold.

8. The method of claim 1 , further comprising generating a plurality of datasets, wherein generating the plurality of datasets comprises:

receiving a first plurality of data requests requesting a first plurality of data units;

loading the first plurality of data units into the first cache unit;

receiving a second plurality of data requests requesting a second plurality of data units;

determining a hit ratio of the first cache unit in responding to the second plurality of data requests;

determining that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit ratio threshold; and

responsive to the determination that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit ratio threshold, storing the first plurality of data units as a first dataset of the plurality of datasets.

9. A non-transitory computer-readable medium comprising instructions that, when executed by a processing device, cause the processing device to:

determine, by the processing device, that a hit ratio is below a hit first ratio threshold associated with a first cache unit and above a second hit ratio threshold associated with a second cache unit, wherein the first hit ratio threshold is different from the second hit ratio threshold; and

responsive to determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit, load a dataset into the first cache unit rather than the second cache unit, wherein the first cache unit and the second cache unit are available to load the dataset.

10. The non-transitory computer-readable medium of claim 9 , the processing device further to:

load a first dataset into the first cache unit and the second cache unit prior to determining that the hit ratio is below the first hit ratio threshold and above the second hit ratio threshold, wherein the dataset loaded into the first cache unit rather than the second cache unit is a second dataset.

11. The non-transitory computer-readable medium of claim 9 , the processing device further to:

determine that the hit ratio is below the second hit ratio threshold associated with the second cache unit; and

responsive to determining that the hit ratio is below the second hit ratio threshold associated with the second cache unit, load a third dataset into the second cache unit.

12. The non-transitory computer-readable medium of claim 9 , the processing device further to:

determine that the hit ratio is below the first hit ratio threshold associated with the first cache unit and equal to the second hit ratio threshold associated with the second cache unit; and

responsive the determination that the hit ratio is below the first hit ratio threshold associated with the first cache unit and equal to the second hit ratio threshold associated with the second cache unit, load the dataset into the first cache unit.

13. The non-transitory computer-readable medium of claim 10 , the processing device further to:

receive an alteration to the first dataset; and

in response to receiving the alteration, load an altered first dataset into a third cache unit.

14. The non-transitory computer-readable medium of claim 9 , wherein to determine that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit the processing device further to:

identify a first number indicative of a number of data requests received by the first cache unit and the second cache unit within a window, each data request requesting a data unit;

identify a second number indicative of a number of the data requests within the window requesting a data unit in a first dataset;

identify the hit ratio of the first cache unit and the second cache unit by dividing the second number by the first number; and

compare the hit ratio to the first hit ratio threshold and the second hit ratio threshold.

15. The non-transitory computer-readable medium of claim 9 , the processing device further to generate a plurality of datasets, wherein to generate the plurality of datasets, the processing device further to:

receive a first plurality of data requests requesting a respective first plurality of data units;

load the first plurality of data units into the first cache unit; receive a second plurality of data requests requesting a respective second plurality of data units;

determine a hit ratio of the first cache unit in responding to the second plurality of data requests;

determine that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit ratio threshold; and

responsive to the determination that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit ratio threshold, store the first plurality of data units as a first dataset of the plurality of datasets.

16. A system comprising: a memory; and

a processing device, operatively coupled to the memory, to:

determine that a hit ratio is below a first hit ratio threshold associated with a first cache unit and above a second hit ratio threshold associated with a second cache unit, wherein the first hit ratio threshold is different from the second hit ratio threshold; and

responsive to determining that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit, load a dataset into the first cache unit rather than the second cache unit, wherein the first cache unit and the second cache unit are available to load the dataset.

17. The system of claim 16 , the processing device further to:

load a first dataset into the first cache unit and the second cache unit prior to determining the that the hit ratio is below the first hit ratio threshold and above the second hit ratio threshold, wherein the dataset loaded into the first cache unit rather than the second cache unit is a second dataset.

18. The system of claim 16 , wherein the processing device is further to:

determine that the hit ratio is below the second hit ratio threshold associated with the second cache unit; and

responsive to the determination that the hit ratio of is below the second hit ratio threshold associated with the second cache unit, load a third dataset into the second cache unit.

19. The system of claim 16 , wherein to determine that the hit ratio is below the first hit ratio threshold associated with the first cache unit and above the second hit ratio threshold associated with the second cache unit, the processing device is to:

identify a first number indicative of a number of data requests received by the first cache unit and the second cache unit within a window, each data request requesting a data unit;

identify a second number indicative of a number of the data requests requesting a data unit in a first dataset within the window;

identify the hit ratio of the first cache unit and the second cache unit by dividing the second number by the first number; and

compare the hit ratio to the first hit ratio threshold and the second hit ratio threshold.

20. The system of claim 16 , the processing device further to generate a plurality of datasets, wherein to generate the plurality of datasets, the processing device further to:

receive a first plurality of data requests requesting a first plurality of data units;

load the first plurality of data units into the first cache unit; receive a second plurality of data requests requesting a second plurality of data units;

determine a hit ratio of the first cache unit in responding to the second plurality of data requests;

determine that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit ratio threshold; and

responsive to the determination that the hit ratio of the first cache unit in responding to the second plurality of data requests is below a dataset generation hit generation threshold, store the first plurality of data units as a first dataset of the plurality of datasets.