Methods and systems for augmentation and feature cache
An augmentation and feature caching subsystem is described for training AI/ML models. In one particular aspect, a method is provided that includes receiving data comprising training examples, one or more augmentation configuration hyperparameters and one or more feature extraction configuration hyperparameters; generating a first key based on one of the training examples and the one or more augmentation configuration hyperparameters; searching a first key-value storage based on the first key; obtaining one or more augmentations based on the search of the first key-value storage; applying the obtained one or more augmentations to the training examples to result in augmented training examples; generating a second key based on one of the augmented training examples and the one or more feature extraction configuration hyperparameters; searching a second key-value storage based on the second key; obtaining one or more features based on the search of the second key-value storage.
1 . A method comprising:
receiving data comprising training examples, one or more augmentation configuration hyperparameters and one or more feature extraction configuration hyperparameters, wherein the training examples, the one or more augmentation configuration hyperparameters and the one or more feature extraction configuration hyperparameters are configured to query, store or retrieve information in a caching subsystem;
generating a first key based on one of the training examples and the one or more augmentation configuration hyperparameters;
searching a first key-value storage in the caching subsystem based on the first key;
obtaining one or more augmentations in response to the search of the first key-value storage;
applying the obtained one or more augmentations to the training examples to result in augmented training examples;
generating a second key based on one of the augmented training examples and the one or more feature extraction configuration hyperparameters;
searching a second key-value storage in the caching subsystem based on the second key;
obtaining one or more features in response to the search of the second key-value storage; and
training a Machine-Learning (ML) model using the obtained one or more features.
2 . The method of claim 1 , further comprising, requesting new augmentations in accordance with the search indicating that no entry in the first key-value storage contains the first key.
3 . The method of claim 1 , further comprising retrieving one or more cached augmentations from the first key-value storage in accordance with the search indicating an entry of the first key-value storage contains the first key.
4 . The method of claim 1 , further comprising requesting new features in accordance with the search indicating that no entry in the second key-value storage contains the second key.
5 . The method of claim 1 , further comprising retrieving one or more cached features from the second key-value storage in accordance with the search indicating an entry of the second key-value storage contains the second key.
6 . The method of claim 1 , wherein the first key is in binary format and comprises a combination of a hash value of the one or more augmentation configuration hyperparameters and a byte-encoded training example; and wherein the second key is in binary format and comprises a combination of a hash value of the one or more feature extraction configuration hyperparameters and a byte-encoded_augmented_training example.
7 . The method of claim 1 , wherein the first and the second key-value storage comprises entries of information, wherein each entry comprises a binary key, a compressed value, and a validity information.
8 . The method of claim 7 , wherein the compressed value comprises augmentations in the first key-value storage, and features in the second key-value storage.
9 . The method of claim 1 , wherein the one or more augmentations obtained from the first key-value storage and the one or more features obtained from the second key-value storage are deterministic.
10 . A computer-program product tangibly embodied in one or more non-transitory machine-readable media, including instructions configured to cause one or more data processors to perform operations comprising:
receiving data comprising training examples, one or more augmentation configuration hyperparameters and one or more feature extraction configuration hyperparameters, wherein the training examples, the one or more augmentation configuration hyperparameters and the one or more feature extraction configuration hyperparameters are configured to query, store or retrieve information in a caching subsystem;
generating a first key based on one of the training examples and the one or more augmentation configuration hyperparameters;
searching a first key-value storage in the caching subsystem based on the first key;
obtaining one or more augmentations in response to the search of the first key-value storage;
applying the obtained one or more augmentations to the training examples to result in augmented training examples;
generating a second key based on one of the augmented training examples and the one or more feature extraction configuration hyperparameters;
searching a second key-value storage in the caching subsystem based on the second key;
obtaining one or more features in response to the search of the second key-value storage; and
training a Machine-Learning (ML) model using the obtained one or more features.
11 . The non-transitory machine-readable media of claim 10 , further comprising requesting new augmentations in accordance with the search indicating that no entry in the first key-value storage contains the first key.
12 . The non-transitory machine-readable media of claim 10 , further comprising retrieving one or more cached augmentations from the first key-value storage in accordance with the search indicating an entry of the first key-value storage contains the first key.
13 . The non-transitory machine-readable media of claim 10 , further comprising requesting new features in accordance with the search indicating that no entry in the second key-value storage contains the second key.
14 . The non-transitory machine-readable media of claim 10 , further comprising retrieving one or more cached features from the second key-value storage in accordance with the search indicating an entry of the second key-value storage contains the second key.
15 . The non-transitory machine-readable media of claim 10 , wherein the first key is in binary format and comprises a combination of a hash value of the one or more augmentation configuration hyperparameters and a byte-encoded training example; and wherein the second key is in binary format and comprises a combination of a hash value of the one or more feature extraction configuration hyperparameters and a byte-encoded_augmented_training example.
16 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving data comprising training examples, one or more augmentation configuration hyperparameters and one or more feature extraction configuration hyperparameters, wherein the training examples, the one or more augmentation configuration hyperparameters and the one or more feature extraction configuration hyperparameters are configured to query, store or retrieve information in a caching subsystem;
generating a first key based on one of the training examples and the one or more augmentation configuration hyperparameters;
searching a first key-value storage in the caching subsystem based on the first key;
obtaining one or more augmentations in response to the search of the first key-value storage;
applying the obtained one or more augmentations to the training examples to result in augmented training examples;
generating a second key based on one of the augmented training examples and the one or more feature extraction configuration hyperparameters;
searching a second key-value storage in the caching subsystem based on the second key;
obtaining one or more features in response to the search of the second key-value storage; and
training a Machine-Learning (ML) model using the obtained one or more features.
17 . The system of claim 16 , wherein the first key is in binary format and comprises a combination of a hash value of the one or more augmentation configuration hyperparameters and a byte-encoded training example; and wherein the second key is in binary format and comprises a combination of a hash value of the one or more feature extraction configuration hyperparameters and a byte-encoded_augmented_training example.
18 . The system of claim 16 , wherein the first and the second key-value storage comprises entries of information, and wherein each entry comprises a binary key, a compressed value, and a validity information.
19 . The system of claim 18 , wherein the compressed value comprises augmentations in the first key-value storage, and features in the second key-value storage.
20 . The system of claim 16 , wherein the one or more augmentations obtained from the first key-value storage and the one or more features obtained from the second key-value storage are deterministic.