Method and system for evicting and reloading a cache for machine learning training data streams

1. A method for managing training data, comprising:

monitoring, by a training data stream manager (TDSM), a cache comprising a plurality of training data examples, each respectively associated with streams of mini-batch sequences scheduled to be transmitted to a machine learning training environment;

making a first determination that a cache eviction is required;

in response to the first determination:

selecting a training data example of the plurality of training data examples;

making a second determination that the training data example is eligible for cache eviction;

in response to the second determination:

evicting the training data example from the cache; and

updating a training data example database entry to indicate that the training data example is evicted from the cache.

2. The method of claim 1 , wherein making the second determination that the training data example is eligible for cache eviction comprises:

making a third determination that a stream reference count associated with the training data example is zero;

in response to the third determination:

making a fourth determination that the training data example is not needed for future scheduled mini-batch sequences currently being processed;

in response to the fourth determination:

making a fifth determination that there are a few remaining mini-batches associated with the mini-batch sequences that comprise the training data example; and

in response to the fifth determination:

making a sixth determination that the training data example does not require regeneration.

3. The method of claim 2 , wherein the stream reference count specifies a number of scheduled mini-batches that comprise the training data example.

4. The method of claim 1 , further comprising:

obtaining a mini-batch entry associated with the next mini-batch to be generated from a mini-batch database;

selecting a training data example identifier associated with the training data example in the mini-batch;

obtaining the training data example database entry;

making a third determination that the training data example database entry indicates that the training data example was evicted from the cache; and

in response to the third determination:

updating the training data example entry to indicate that the training data example is invalid.

5. The method of claim 4 , further comprising:

in response to updating the training data example entry to indicate that the training data example is invalid:

changing the training data example entry to indicate that the training data example is loading;

loading training data associated with the training data example using a stream specification associated with the mini-batch entry;

performing processing on the training data to generate the training data example;

installing the training data example in the cache; and

updating the training data example entry to indicate that training data example is cached.

6. The method of claim 1 , wherein a mini-batch sequence of the mini-batch sequences comprises:

a plurality of mini-batches;

end of epoch messages; and

an end of stream message.

7. The method of claim 6 , wherein a mini-batch of the mini-batch sequence comprises a randomly sampled portion of training data examples.

8. A system for managing training data, comprising:

a cache; and

a training data stream manager (TDSM), comprising a processor and memory, programmed to:

monitor the cache comprising a plurality of training data examples, each respectively associated with streams of mini-batch sequences scheduled to be transmitted to a machine learning training environment;

make a first determination that a cache eviction is required;

in response to the first determination:

select a training data example of the plurality of training data examples;

make a second determination that the training data example is eligible for cache eviction;

in response to the second determination:

evict the training data example from the cache; and

update a training data example database entry to indicate that the training data example is evicted from the cache.

9. The system of claim 8 , wherein making the second determination that the training data example is eligible for cache eviction comprises:

making a third determination that a stream reference count associated with the training data example is zero;

in response to the third determination:

making a fourth determination that the training data example is not needed for future scheduled mini-batch sequences currently being processed;

in response to the fourth determination:

making a fifth determination that there are a few remaining mini-batches associated with the mini-batch sequences that comprise the training data example; and

in response to the fifth determination:

making a sixth determination that the training data example does not require regeneration.

10. The system of claim 9 , wherein the stream reference count specifies a number of scheduled mini-batches that comprise the training data example.

11. The system of claim 8 , wherein the TDSM is further programmed to:

obtain a mini-batch entry associated with the next mini-batch to be generated from a mini-batch database;

select a training data example identifier associated with the training data example in the mini-batch;

obtain the training data example database entry;

make a third determination that the training data example database entry indicates that the training data example was evicted from the cache; and

in response to the third determination:

update the training data example entry to indicate that the training data example is invalid.

12. The system of claim 11 , wherein the TDSM is further programmed to:

in response to updating the training data example entry to indicate that the training data example is invalid:

change the training data example entry to indicate that the training data example is loading;

load training data associated with the training data example using a stream specification associated with the mini-batch entry;

perform processing on the training data to generate the training data example;

install the training data example in the cache; and

update the training data example entry to indicate that training data example is cached.

13. The system of claim 8 , wherein a mini-batch sequence of the mini-batch sequences comprises:

a plurality of mini-batches;

end of epoch messages; and

an end of stream message.

14. The system of claim 13 , wherein a mini-batch of the mini-batch sequence comprises a randomly sampled portion of training data examples.

15. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing training data, the method comprising:

monitoring, by a training data stream manager (TDSM), a cache comprising a plurality of training data examples, each respectively associated with streams of mini-batch sequences scheduled to be transmitted to a machine learning training environment;

making a first determination that a cache eviction is required;

in response to the first determination:

selecting a training data example of the plurality of training data examples;

making a second determination that the training data example is eligible for cache eviction;

in response to the second determination:

evicting the training data example from the cache; and

updating a training data example database entry to indicate that the training data example is evicted from the cache.

16. The non-transitory computer readable medium of claim 15 , wherein making the second determination that the training data example is eligible for cache eviction comprises:

making a third determination that a stream reference count associated with the training data example is zero;

in response to the third determination:

making a fourth determination that the training data example is not needed for future scheduled mini-batch sequences currently being processed;

in response to the fourth determination:

making a fifth determination that there are a few remaining mini-batches associated with the mini-batch sequences that comprise the training data example; and

in response to the fifth determination:

making a sixth determination that the training data example does not require regeneration.

17. The non-transitory computer readable medium of claim 16 , wherein the stream reference count specifies a number of scheduled mini-batches that comprise the training data example.

18. The non-transitory computer readable medium of claim 15 , wherein the method further comprising:

obtaining a mini-batch entry associated with the next mini-batch to be generated from a mini-batch database;

selecting a training data example identifier associated with the training data example in the mini-batch;

obtaining the training data example database entry;

making a third determination that the training data example database entry indicates that the training data example was evicted from the cache; and

in response to the third determination:

updating the training data example entry to indicate that the training data example is invalid.

19. The non-transitory computer readable medium of claim 18 , wherein the method further comprising:

in response to updating the training data example entry to indicate that the training data example is invalid:

changing the training data example entry to indicate that the training data example is loading;

loading training data associated with the training data example using a stream specification associated with the mini-batch entry;

performing processing on the training data to generate the training data example;

installing the training data example in the cache; and

updating the training data example entry to indicate that training data example is cached.

20. The non-transitory computer readable medium of claim 15 , wherein a mini-batch sequence of the mini-batch sequences comprises:

a plurality of mini-batches;

end of epoch messages; and

an end of stream message.