Method and system of retrieving assets from personalized asset libraries
A system and method and for retrieving assets from a personalized asset library includes receiving a search query for searching for assets in one or more asset libraries, the one or more asset libraries including a personalized asset library; encoding the search query into embedding representations via a trained query representation machine-learning (ML) model; comparing, via a matching unit, the query embedding representations to a plurality of asset representations, each of the plurality of asset representations being a representation of one of the plurality of candidate assets; identifying, based on the comparison, at least one of the plurality of the candidate assets as a search result for the search query; and providing the identified plurality of candidate assets for display as the search result. The plurality of asset representations for the one or more assets in the personalized content library are generated automatically without human labeling.
1. An improved data processing system for personalized search and retrieval of assets comprising:
a processor; and
a memory in communication with the processor, the memory comprising executable instructions that, when executed by the processor, cause the improved data processing system to perform functions of:
receiving a search query via a user interface screen of an application for searching for one or more assets for inserting into a document that is being created via the application, the user interface screen providing a first user interface element for selecting a type of visual asset to search for, a second user interface element for selecting one or more asset libraries to search for assets, and a third user interface element for entering the search query, the one or more assets being among assets that are stored in a plurality of asset libraries, the plurality of asset libraries including a personalized asset library and a global asset library, each of the plurality of asset libraries including a plurality of candidate assets;
converting the search query into one or more vector embeddings in a vector space, via a trained query representation machine-learning (ML) model;
comparing, via a matching unit, the one or more vector embeddings to a plurality of asset representations to generate a similarity score between the one or more vector embeddings and at least two of the plurality of asset representations, each of the plurality of asset representations being a representation of one of the plurality of candidate assets in a vector embedding space and each of the plurality of asset representations being stored in a content index library that corresponds to one of the plurality of asset libraries;
identifying, based on the similarity score multiple of the plurality of candidate assets as a search result for the search query, the search result combining candidate assets that are stored in different asset libraries;
ranking the search result from the different asset libraries based on user preferences; and
providing at least one of the plurality of the candidate assets for display as the search result, wherein:
the plurality of asset representations for the one or more assets in the personalized asset library are generated automatically in a zero-shot manner,
the trained query representation ML model is trained using a training dataset,
the training dataset is updated to include supplemental training data, and
the trained query representation ML model is regenerated using the updated training dataset to improve the trained query representation ML model.
2. The improved data processing system of claim 1 , wherein the personalized asset library includes at least one of an enterprise asset library and a consumer asset library.
3. The improved data processing system of claim 1 , wherein the personalized asset library is provided to a content representation engine for converting the plurality of candidate assets in the personalized asset library to asset indices.
4. The improved data processing system of claim 3 , wherein converting the personalized asset library to asset indices occurs in an offline stage.
5. The improved data processing system of claim 3 , wherein converting the personalized asset library to asset indices is done by a local content representation engine stored locally at a client device.
6. The improved data processing system of claim 1 , wherein:
the search query includes a selection of the global asset library and the personalized asset library, and
the matching unit combines search results from the global asset library and the personalized asset library.
7. The improved data processing system of claim 1 , wherein ranking the search result is performed by a local ranking unit.
8. A method for personalized search and retrieval of one or more assets comprising:
receiving a search query via a user interface screen of an application for searching for one or more assets for inserting into a document that is being created via the application, the user interface screen providing a first user interface element for selecting a type of visual asset to search for, a second user interface element for selecting one or more asset libraries to search for assets, and a third user interface element for entering the search query, the one or more assets being among assets that are stored in a plurality of asset libraries, the plurality of asset libraries including a personalized asset library and a global asset library, each of the plurality of asset libraries including a plurality of candidate assets;
converting the search query into one or more vector embeddings in a vector space, via a trained query representation machine-learning (ML) model;
comparing, via a matching unit, the one or more vector embeddings to a plurality of asset representations to generate a similarity score between the one or more vector embeddings and at least two of the plurality of asset representations, each of the plurality of asset representations being a representation of one of the plurality of candidate assets in a vector embedding space and each of the plurality of asset representations being stored in a content index library that corresponds to one of the plurality of asset libraries;
identifying, based on the similarity score multiple of the plurality of the candidate assets as a search result for the search query, the search result combining candidate assets that are stored in different asset libraries;
ranking the search result from the different asset libraries based on user preferences; and
providing at least one of the plurality of the candidate assets for display as the search result,
wherein:
the plurality of asset representations for the one or more assets in the personalized asset library are generated automatically without human labeling,
the trained query representation ML model is trained using a training dataset, the training dataset is updated to include supplemental training data, and
the trained query representation ML model is regenerated using the updated training dataset to improve the trained query representation ML model.
9. The method of claim 8 , wherein the personalized asset library is provided to a content representation engine for converting the plurality of candidate assets in the personalized asset library to asset indices.
10. The method of claim 9 , wherein converting the personalized asset library to asset indices occurs in an offline stage.
11. The method of claim 9 , wherein converting the personalized asset library to asset indices is done by a local content representation engine stored locally at a client device.
12. The method of claim 8 , wherein:
the search query includes a selection of the global asset library and the personalized asset library, and
the matching unit combines search results from the global asset library and the personalized asset library.
13. The method of claim 8 , wherein ranking the search result is performed by a local ranking unit.
14. A non-transitory computer readable medium on which are stored instructions that, when executed, cause a programmable device to perform functions of:
receiving a search query via a user interface screen of an application for searching for one or more assets for inserting into a document that is being created via the application, the user interface screen providing a first user interface element for selecting a type of visual asset to search for, a second user interface element for selecting one or more asset libraries to search for assets, and a third user interface element for entering the search query, the one or more assets being among assets that are stored in a plurality of asset libraries, the plurality of asset libraries including a personalized asset library and a global asset library, each of the plurality of asset libraries including a plurality of candidate assets;
converting the search query into one or more vector embeddings in a vector space, via a trained query representation machine-learning (ML) model;
comparing, via a matching unit, the one or more vector embeddings to a plurality of asset representations to generate a similarity score between the one or more vector embeddings and at least two of the plurality of asset representations, each of the plurality of asset representations being a representation of one of the plurality of candidate assets in a vector embedding space and each of the plurality of asset representations being stored in a content index library that corresponds to one of the plurality of asset libraries;
identifying, based on the similarity score multiple of the plurality of the candidate assets as a search result for the search query, the search result combining candidate assets that are stored in different asset libraries;
providing at least one of the plurality of the candidate assets for display as the search result, wherein:
the plurality of asset representations for the one or more assets in the personalized asset library are generated automatically without human labeling,
the trained query representation ML model is trained using a training dataset,
the training dataset is updated to include supplemental training data, and
the trained query representation ML model is regenerated using the updated training dataset to improve the trained query representation ML model.
15. The non-transitory computer readable medium of claim 14 , wherein the personalized asset library is provided to a content representation engine for converting the plurality of candidate assets in the personalized asset library to asset indices.
16. The non-transitory computer readable medium of claim 14 ,
wherein:
the search query includes a selection of the global asset library and the personalized asset library, and
the matching unit combines search results from the global asset library and the personalized asset library.