Contextual local image recognition dataset
A contextual local image recognition module of a device retrieves a primary content dataset from a server and then generates and updates a contextual content dataset based on an image captured with the device. The device stores the primary content dataset and the contextual content dataset. The primary content dataset comprises a first set of images and corresponding virtual object models. The contextual content dataset comprises a second set of images and corresponding virtual object models retrieved from the server.
1. A method comprising:
determining, by a computing device, that a first image captured by an optical sensor of the computing device is not included in a primary content dataset stored in a memory of the computing device, the primary content dataset including a subset of images included in a full content dataset stored by a remote server, the primary content dataset having been received by the computing device from the remote server;
in response to determining that the first image is not included in the primary content dataset stored in the memory of the computing device, transmitting the first image to the remote server, causing the remote server to compare the first image to images included in a full content dataset maintained by the remote server, wherein each image in the full content data set is stored by the remote server along with corresponding virtual content; and
receiving, from the remote server, virtual content corresponding to a second image from the full content dataset that matches the first image captured by an optical sensor of the computing device.
2. The method of claim 1 , further comprising:
presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of a first physical object that is being concurrently presented on the display of the computing device.
3. The method of claim 1 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the method further comprising:
presenting the virtual content at a position on the transparent display that overlaps with the user's view of a first physical object captured in the first image.
4. The method of claim 1 , further comprising:
receiving the primary content dataset from the remote server, the primary content dataset including the subset of images from the full content dataset that were most commonly captured by a plurality of computing devices; and
storing the primary content dataset in a local memory of the computing device.
5. The method of claim 4 , further comprising:
receiving, from the remote server, a set of virtual content corresponding to the primary content dataset.
6. The method of claim 1 , further comprising:
capturing a second image of a second physical object, the second physical object being different than a first physical object captured in the first image;
comparing the second image of the second physical object to images included in the primary content dataset, yielding a comparison;
determining, based on the comparison, that the second image of the second physical object matches a third image from the primary content dataset; and
presenting, on a display of the computing device, virtual content corresponding to the third image from the primary content dataset.
7. The method of claim 1 , wherein determining that the first image is not included in the primary content dataset comprises:
identifying a visual reference captured in the first image; and
comparing the visual reference captured in the first image to visual references captured in the images included in the primary content dataset.
8. A computing device comprising:
one or more computer processors; and
one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the computing device to perform operations comprising:
determining that a first image captured by an optical sensor of the computing device is not included in a primary content dataset stored in a memory of the computing device, the primary content dataset including a subset of images included in a full content dataset stored by a remote server, the primary content dataset having been received by the computing device from the remote server;
in response to determining that the first image is not included in the primary content dataset stored in the memory of the computing device, transmitting the first image to the remote server, causing the remote server to compare the first image to images included in a full content dataset maintained by the remote server, wherein each image in the full content data set is stored by the remote server along with corresponding virtual content; and
receiving, from the remote server, virtual content corresponding to a second image from the full content dataset that matches the first image captured by an optical sensor of the computing device.
9. The computing device of claim 8 , the operations further comprising:
presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of a first physical object that is being concurrently presented on the display of the computing device.
10. The computing device of claim 8 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the method further comprising:
presenting the virtual content at a position on the transparent display that overlaps with the user's view of a first physical object captured in the first image.
11. The computing device of claim 8 , the operations further comprising:
receiving the primary content dataset from the remote server, the primary content dataset including the subset of images from the full content dataset that were most commonly captured by a plurality of computing devices; and
storing the primary content dataset in a local memory of the computing device.
12. The computing device of claim 11 , the operations further comprising:
receiving, from the remote server, a set of virtual content corresponding to the primary content dataset.
13. The computing device of claim 8 , the operations further comprising:
capturing a second image of a second physical object, the second physical object being different than a first physical object captured in the first image;
comparing the second image of the second physical object to images included in the primary content dataset, yielding a comparison;
determining, based on the comparison, that the second image of the second physical object matches a third image from the primary content dataset; and
presenting, on a display of the computing device, virtual content corresponding to the third image from the primary content dataset.
14. The computing device of claim 8 , wherein determining that the first image is not included in the primary content dataset comprises:
identifying a visual reference captured in the first image; and
comparing the visual reference captured in the first image to visual references captured in the images included in the primary content dataset.
15. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a computing device, cause the computing device to perform operations comprising:
determining that a first image captured by an optical sensor of the computing device is not included in a primary content dataset stored in a memory of the computing device, the primary content dataset including a subset of images included in a full content dataset stored by a remote server, the primary content dataset having been received by the computing device from the remote server;
in response to determining that the first image is not included in the primary content dataset stored in the memory of the computing device, transmitting the first image to the remote server, causing the remote server to compare the first image to images included in a full content dataset maintained by the remote server, wherein each image in the full content data set is stored by the remote server along with corresponding virtual content; and
receiving, from the remote server, virtual content corresponding to a second image from the full content dataset that matches the first image captured by an optical sensor of the computing device.
16. The non-transitory computer-readable medium of claim 15 , the operations further comprising:
presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of a first physical object that is being concurrently presented on the display of the computing device.
17. The non-transitory computer-readable medium of claim 15 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the method further comprising:
presenting the virtual content at a position on the transparent display that overlaps with the user's view of a first physical object captured in the first image.
18. The non-transitory computer-readable medium of claim 15 , the operations further comprising:
receiving the primary content dataset from the remote server, the primary content dataset including the subset of images from the full content dataset that were most commonly captured by a plurality of computing devices;
receiving, from the remote server, a set of virtual content corresponding to the primary content dataset; and
storing the primary content dataset and the set of virtual content in a local memory of the computing device.
19. The non-transitory computer-readable medium of claim 15 , the operations further comprising:
capturing a second image of a second physical object, the second physical object being different than a first physical object captured in the first image;
comparing the second image of the second physical object to images included in the primary content dataset, yielding a comparison;
determining, based on the comparison, that the second image of the second physical object matches a third image from the primary content dataset; and
presenting, on a display of the computing device, virtual content corresponding to the third image from the primary content dataset.
20. The non-transitory computer-readable medium of claim 15 , wherein determining that the first image is not included in the primary content dataset comprises:
identifying a visual reference captured in the first image; and
comparing the visual reference captured in the first image to visual references captured in the images included in the primary content dataset.