Retrieval aware embedding
Systems and methods for information retrieval are described. Embodiments generate a dense embedding for each of a plurality of media objects to be searched, generate a sparse embedding for each of the media objects using an encoder that takes the dense embedding as an input, wherein the sparse embedding satisfies a sparsity constraint that is applied to at least one layer of the encoder during training, and perform a search on the plurality of media objects based at least in part on the sparse embedding.
1. A method for information retrieval, comprising:
generating a dense image embedding for each of a plurality of media objects to be searched;
generating a sparse image embedding with a higher number of dimensions than the dense image embedding for each of the media objects using an encoder that takes the dense image embedding as an input, wherein the sparse image embedding is generated using an activation function including a sparsity constraint that limits a number of non-zero parameters at a hidden layer of the encoder;
performing a search on the plurality of media objects based at least in part on the sparse image embedding; and
retrieving a matching object from the plurality of media objects based on the search.
2. The method of claim 1 , further comprising:
receiving a query object of the plurality of media objects;
generating a sparse query embedding for the query object, wherein the search is based on the sparse query embedding;
identifying the matching object similar to the query object based on the search; and
presenting the matching object to a user.
3. The method of claim 1 , further comprising:
receiving a text query;
identifying the matching object corresponding to the text query based on the search; and
presenting the matching object to a user.
4. The method of claim 1 , further comprising:
storing the media objects and the sparse image embedding in a database;
identifying a stored media object based on the search; and
retrieving the stored media object from the database.
5. The method of claim 1 , wherein:
at least one layer of the encoder comprises an intermediate layer of an auto-encoder during a training phase.
6. The method of claim 1 , wherein:
the sparsity constraint comprises a limit on a number of non-zero parameters in at least one layer of the encoder.
7. The method of claim 1 , wherein:
the media objects comprise image files.
8. The method of claim 1 , wherein:
the media objects comprise at least two media types from a set comprising audio files, video files, image files, and text files.
9. A non-transitory computer readable medium storing code for information retrieval, the code comprising instructions executable by at least one processor to:
generate a dense image embedding for a plurality of media objects;
generate a sparse image embedding with a higher number of dimensions than the dense image embedding for each of the media objects using an encoder that takes the dense image embedding as an input, wherein the sparse image embedding is generated using an activation function including a sparsity constraint that limits a number of non-zero parameters at a hidden layer of the encoder; and
perform a search of the plurality of media objects based at least in part on the sparse image embedding and retrieve a matching object from the plurality of media objects based on the search.
10. The non-transitory computer readable medium of claim 9 , wherein:
at least one layer of the encoder comprises an intermediate layer of an auto-encoder during a training phase.
11. The non-transitory computer readable medium of claim 9 , wherein:
the sparsity constraint comprises a limit on a number of non-zero parameters in at least one layer of the encoder.
12. The non-transitory computer readable medium of claim 9 , the code further comprising instructions executable by the at least one processor to:
store the plurality of media objects.
13. The non-transitory computer readable medium of claim 9 , the code further comprising instructions executable by the at least one processor to:
generate an additional dense embedding for additional media objects having a different media type from the plurality of media objects.
14. An apparatus comprising:
a processor; and
a memory including instructions executable by the processor to:
generate a dense image embedding for each of a plurality of media objects to be searched;
generate a sparse image embedding with a higher number of dimensions than the dense image embedding for each of the media objects using an encoder that takes the dense image embedding as an input, wherein the sparse image embedding is generated using an activation function including a sparsity constraint that limits a number of non-zero parameters at a hidden layer of the encoder;
perform a search on the plurality of media objects based at least in part on the sparse image embedding; and
retrieve a matching object from the plurality of media objects based on the search.
15. The apparatus of claim 14 , further comprising instructions executable by the processor to:
receive a query object of the plurality of media objects;
generate a sparse query embedding for the query object, wherein the search is based on the sparse query embedding;
identify the matching object similar to the query object based on the search; and
present the matching object to a user.
16. The apparatus of claim 14 , further comprising instructions executable by the processor to:
receive a text query;
identify the matching object corresponding to the text query based on the search; and
present the matching object to a user.
17. The apparatus of claim 14 , further comprising instructions executable by the processor to:
store the media objects and the sparse image embedding in a database;
identify a stored media object based on the search; and
retrieve the stored media object from the database.
18. The apparatus of claim 14 , wherein:
the sparsity constraint comprises a limit on a number of non-zero parameters in at least one layer of the encoder.