Computing systems and methods for retrieving data from a library for use by downstream systems
The present disclosure provides computing systems and methods for cataloging, retrieving, and/or organizing user-generated content associated with objects. Aspects of the disclosure are directed to a systems and methods which utilize computers to enable users to interact with libraries of user-generated content associated with cataloged objects. For example, a user can capture one or more images of a real-world object, label or otherwise annotate the object with various types of user-generated content and organize the object and its associated content into one or more libraries. The user-generated content can then be provided to other users upon the receipt of images of the same object or an object displaying similar features.
1 . A computing system for retrieving data associated with objects from a library for providing to additional systems, applications, devices, or components, the computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
receiving a dataset that comprises one or more images that depict an object;
processing, using an artificial neural network, the dataset to generate an embedding associated with the object;
accessing one or more libraries of a plurality of libraries of content, wherein each of the plurality of libraries comprises one or more cataloged embeddings respectively associated with one or more cataloged objects, wherein the one or more cataloged embeddings are grouped into the library based at least in part on similarities associated with one or more corresponding embedding characteristics of the one or more cataloged embeddings, wherein each cataloged embedding has been generated from one or more images of the associated cataloged object, wherein one or more items of content are associated with each cataloged object, and wherein the one or more libraries are a subset of the plurality of libraries;
identifying, based at least in part on a comparison of the embedding with one or more cataloged embeddings included in the one or more libraries, a closest cataloged embedding of the one or more cataloged embeddings included in the one or more libraries, wherein the closest cataloged embedding has been generated using the artificial neural network from user-supplied imagery in the one or more libraries; and
providing a signal to one or more additional systems, applications, devices, or components based on identifying the closest cataloged embedding of the one or more cataloged embeddings included in the one or more libraries, wherein the signal provides the closest cataloged embedding, the cataloged object associated with the closest cataloged embedding, or content associated with the cataloged object that is associated with the closest cataloged embedding.
2 . The computing system of claim 1 , wherein the one or more additional systems, applications, devices, or components trigger an action or output based on the signal.
3 . The computing system of claim 1 , wherein the signal:
identifies the cataloged object associated with the closest cataloged embedding; or
provides content associated with the cataloged object that is associated with the closest cataloged embedding.
4 . The computing system of claim 1 , wherein the content comprises:
a textual annotation associated with the cataloged object that is associated with the closest cataloged embedding; or
user-generated content submitted by a user associated with the cataloged object that is associated with the closest cataloged embedding.
5 . The computing system of claim 1 , wherein providing the signal to the one or more additional systems, applications, devices, or components comprises providing the signal to the one or more additional systems, applications, devices, or components via a pre-defined application programming interface,
further comprising:
causing, by providing the signal via the pre-defined application programming interface, presentation of the content associated with the closest cataloged embedding or the content associated with the cataloged object, by one or more additional systems, applications, devices, or components.
6 . The computing system of claim 1 , wherein the one or more additional systems, applications, devices, or components comprise a computing device that is physically remote from the computing system and wherein said providing the signal is performed over a wireless network.
7 . The computing system of claim 1 , wherein:
the dataset further comprises data descriptive of a current location of a user at a time the one or more images were captured; and
accessing the one or more libraries of the plurality of libraries comprises:
selecting the one or more libraries from the plurality of libraries based at least in part on the current location of the user at the time the one or more images were captured; or
downloading at least one of the one or more libraries.
8 . The computing system of claim 1 , wherein accessing the one or more libraries of the plurality of libraries of content comprises:
receiving a user input that selects the one or more libraries of the plurality of libraries of content; and
in response to the user input, accessing the one or more libraries from a data storage location.
9 . The computing system of claim 1 , wherein:
the plurality of libraries of content are respectively associated with a plurality of other users and each of the other users is able to control a set of access permissions for the respective library associated with such other user; and
accessing the one or more libraries of the plurality of libraries of content comprises accessing only libraries for which the user has been granted permission to access.
10 . The computing system of claim 1 , wherein the one or more processors and the one or more non-transitory computer-readable media are included in a user computing device that implements:
an on-device content platform that serves, via an application programming interface, content to one or more client applications included on the user computing device that are distinct and separate from the on-device content platform; or
another application that includes the on-device content platform that serves the content to the other application.
11 . The computing system of claim 1 , wherein processing the dataset to generate the embedding associated with the object comprises using an object-class-specific embedding model to generate the embedding, wherein the object-class-specific embedding model is specific to a class of the object.
12 . The computing system of claim 1 , wherein the operations comprise:
identifying a subset of user-selected libraries as the one or more libraries; and
in response to the identifying of the subset of the user-selected libraries as the one or more libraries, accessing the subset of user-selected libraries and performing the comparison of the embedding with the one or more cataloged embeddings.
13 . The computing system of claim 1 , wherein the one or more cataloged embeddings are grouped into the library using a learning algorithm based at least in part on the similarities.
14 . The computing system of claim 1 , wherein accessing the one or more libraries comprises searching the plurality of libraries of content with a first filter to identify the one or more libraries and then accessing the one or more libraries, and
wherein identifying the closest cataloged embedding comprises searching the one or more libraries with a second filter to identify a closest library and then identifying, based at least in part on a comparison of the embedding with a subset of the one or more cataloged embeddings included in the closest library, the closest cataloged embedding.
15 . A computer implemented method for retrieving data associated with objects from a library for providing to additional systems, applications, devices, or components, the method comprising:
receiving, by one or more computing devices, a dataset that comprises one or more images captured by a user that depict an object;
processing, by one or more computing devices using a neural network, the dataset to generate an embedding associated with the object;
accessing, by the one or more computing devices, one or more libraries of a plurality of libraries, wherein each of the plurality of libraries comprises one or more cataloged embeddings respectively associated with one or more cataloged objects, wherein the one or more cataloged embeddings are grouped into the library based at least in part on similarities associated with one or more corresponding embedding characteristics of the one or more cataloged embeddings, wherein each cataloged embedding has been generated from one or more images of the associated cataloged object, wherein one or more items of content are associated with each embedding, and wherein the one or more libraries are a subset of the plurality of libraries; and
associating, by one or more computing devices, the embedding with one of the cataloged objects based at least in part on a cataloged embedding associated with the cataloged object, wherein the cataloged embedding has been generated using the neural network from user-supplied imagery in the one or more libraries, wherein associating the embedding with one of the cataloged objects produces a new cataloged embedding that is accessible by the one or more computing devices.
16 . The computer implemented method of claim 15 , wherein the one or more computing devices comprise a user device, wherein the user-supplied imagery is supplied from the user device and stored in the one or more libraries, and wherein at least one of actions of receiving, processing, accessing, and associating are performed on the user device.
17 . The computer implemented method of claim 15 , further comprising detecting, by one or more computing devices, an item of inappropriate content using a Spam filter.
18 . The computer implemented method of claim 15 , wherein associating the embedding with one of the cataloged objects comprises searching a hierarchical database, and wherein the hierarchical database includes the cataloged embeddings.
19 . A computing system for retrieving data associated with objects from a library for providing to additional systems, applications, devices, or components, comprising:
one or more data storage devices that store a plurality of libraries of content, wherein each of the plurality of libraries comprises one or more cataloged embeddings respectively associated with one or more cataloged objects, wherein the one or more cataloged embeddings are grouped into the library based at least in part on similarities associated with one or more corresponding embedding characteristics of the one or more cataloged embeddings, wherein each cataloged embedding has been generated from one or more images of the associated cataloged object captured by one or more users, wherein one or more items of content are associated with each embedding; and
one or more computing devices that implement a content platform that:
receives imagery of an object from a client;
uses an artificial neural network to generate an embedding for the object;
searches one or more of the plurality of libraries to identify a cataloged object that has a cataloged embedding that is closest to the embedding for the object, wherein the cataloged embedding has been generated using the artificial neural network from user-supplied imagery in the one or more of the plurality of libraries; and
serves to the client at least one item of content associated with the identified cataloged object.