Method and system for managing embeddings and serving recommendations in real-time
The disclosed embodiments describe techniques for isolating and managing models via versioning. In one embodiment, a method is disclosed comprising reading a configuration document associated with a first model; incrementing an internal write version of the configuration document; storing the internal write version in the configuration document; generating documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field; and uploading the documents to a serving system, causing the serving system to replace the first model with the second model.
1. A method comprising:
reading, by a processing device, a configuration document associated with a first model;
incrementing, by the processing device, an internal write version of the configuration document;
storing, by the processing device, the internal write version in the configuration document;
generating, by the processing device, documents belonging to a second model such that respective document identifiers of the documents include a next external version in a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
uploading, by the processing device, the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.
2. The method of claim 1 , further comprising:
updating the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.
3. The method of claim 2 , further comprising:
reading the configuration document to obtain the internal read version; and
filtering accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.
4. The method of claim 1 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.
5. The method of claim 1 , further comprising:
detecting and discarding obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.
6. The method of claim 1 , further comprising cycling the external version of the document identifiers through a range of numbers.
7. The method of claim 6 , wherein the range of numbers comprises 0 and 1.
8. A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining steps of:
reading a configuration document associated with a first model;
incrementing an internal write version of the configuration document;
storing the internal write version in the configuration document;
generating documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
uploading the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.
9. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:
updating the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.
11. The non-transitory computer-readable storage medium of claim 10 , the steps further comprising:
reading the configuration document to obtain the internal read version; and
filtering accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.
12. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:
detecting and discarding obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.
13. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising cycling the external version of the document identifiers through a range of numbers.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the range of numbers comprises 0 and 1.
15. A device comprising:
a processor configured to:
read a configuration document associated with a first model;
increment an internal write version of the configuration document;
store the internal write version in the configuration document;
generate documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
upload the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.
16. The device of claim 15 , the processor further configured to:
update the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.
17. The device of claim 15 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.
18. The device of claim 17 , the processor further configured to:
read the configuration document to obtain the internal read version; and
filter accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.
19. The device of claim 15 , the processor further configured to:
detect and discard obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.
20. The device of claim 15 , the processor further configured to cycle the external version of the document identifiers through a range of numbers.