Granted Patent B2

US 10,210,663 · App. 15/444,115 · Granted Feb 19, 2019

Inventor: Brian Mullins (Altadena, CA)

Assignee: DAQRI, LLC

G06T19/006G06F1/1686G06F3/005G06F3/147G06F17/3028G06F17/30244G06F17/30256G06F17/30876G06K9/00671G06T1/0007H04N1/00244G06F3/011G06T2200/04H04N2201/0084

Abstract

Claims (69)

1. A method comprising:

capturing, by a computing device, an image of a first physical object, the image captured using an optical sensor of the computing device;

comparing, by the computing device, the image to images included in a primary content dataset maintained by the computing device, yielding a comparison, the primary content dataset received by the computing device from a remote server, wherein the primary content dataset is a subset of a full content dataset maintained by the remote server, and each image in the primary content dataset is stored by the computing device along with corresponding virtual content;

determining, based on the comparison, that the primary content dataset maintained by the computing device does not include the image captured by the computing device;

in response to determining that the primary content dataset does not include the image captured by the computing device, transmitting the image to the remote server, causing the remote server to compare the image to images included in the full content dataset maintained by the remote server, wherein each image in the full content data set corresponding to is stored by the remote server along with corresponding virtual content; and

receiving, from the remote server, virtual content corresponding to an image from the full content dataset that matches the image captured by the computing device.

2. The method of claim 1 , further comprising:

presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of the first physical object that is being concurrently presented on the display of the computing device.

3. The method of claim 1 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the method further comprising:

presenting the virtual content at a position on the transparent display that overlaps with the user's view of the first physical object.

4. The method of claim 1 , further comprising:

receiving the primary content dataset from the remote server, the primary content dataset including images from the full content dataset that were most commonly captured by a plurality of computing devices; and

storing the primary content dataset in a local memory of the computing device.

5. The method of claim 4 , further comprising:

receiving, from the remote server, a set of virtual content corresponding to the primary content dataset.

6. The method of claim 1 , further comprising:

capturing an image of a second physical object, the second physical object being different than the first physical object;

comparing the image of the second physical object to images included in the primary content dataset, yielding a second comparison;

determining, based on the second comparison, that the image of the second physical object matches a first image from the primary content dataset; and

presenting, on a display of the computing device, virtual content corresponding to the first image from the primary content dataset.

7. The method of claim 1 , wherein comparing the image to images included in the primary content dataset comprises:

identifying a visual reference captured in the image; and

comparing the visual reference captured in the image to visual references captured in the images included in the primary content dataset.

8. A computing device comprising:

one or more computer processors; and

one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the computing device to perform operations comprising:

capturing an image of a first physical object, the image captured using an optical sensor of the computing device;

comparing the image to images included in a primary content dataset maintained by the computing device, yielding a comparison, the primary content dataset received by the computing device from a remote server, wherein the primary content dataset is a subset of a full content dataset maintained by the remote server, and each image in the primary content dataset is stored by the computing device along with corresponding virtual content;

determining, based on the comparison, that the primary content dataset maintained by the computing device does not include the image captured by the computing device;

in response to determining that the primary content dataset does not include the image captured by the computing device, transmitting the image to the remote server, causing the remote server to compare the image to images included in the full content dataset maintained by the remote server, wherein each image in the full content data set is stored by the remote server along with corresponding virtual content; and

receiving, from the remote server, virtual content corresponding to an image from the full content dataset that matches the image captured by the computing device.

9. The computing device of claim 8 , the operations further comprising:

presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of the first physical object that is being concurrently presented on the display of the computing device.

10. The computing device of claim 8 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the operations further comprising:

presenting the virtual content at a position on the transparent display that overlaps with the user's view of the first physical object.

11. The computing device of claim 8 , the operations further comprising:

receiving the primary content dataset from the remote server, the primary content dataset including images from the full content dataset that were most commonly captured by a plurality of computing devices; and

storing the primary content dataset in a local memory of the computing device.

12. The computing device of claim 11 , the operations further comprising:

receiving, from the remote server, a set of virtual content corresponding to the primary content dataset.

13. The computing device of claim 8 , the operations further comprising:

capturing an image of a second physical object, the second physical object being different than the first physical object;

comparing the image of the second physical object to images included in the primary content dataset, yielding a second comparison;

determining, based on the second comparison, that the image of the second physical object matches a first image from the primary content dataset; and

presenting, on a display of the computing device, virtual content corresponding to the first image from the primary content dataset.

14. The computing device of claim 8 , wherein comparing the image to images included in the primary content dataset comprises:

identifying a visual reference captured in the image; and

comparing the visual reference captured in the image to visual references captured in the images included in the primary content dataset.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a computing device, cause the computing device to perform operations comprising:

capturing an image of a first physical object, the image captured using an optical sensor of the computing device;

comparing the image to images included in a primary content dataset maintained by the computing device, yielding a comparison, the primary content dataset received by the computing device from a remote server, wherein the primary content dataset is a subset of a full content dataset maintained by the remote server, and each image in the primary content dataset is stored by the computing device along with corresponding virtual content;

determining, based on the comparison, that the primary content dataset maintained by the computing device does not include the image captured by the computing device;

in response to determining that the primary content dataset does not include the image captured by the computing device, transmitting the image to the remote server, causing the remote server to compare the image to images included in the full content dataset maintained by the remote server, wherein each image in the full content data set is stored by the remote server along with corresponding virtual content; and

receiving, from the remote server, virtual content corresponding to an image from the full content dataset that matches the image captured by the computing device.

16. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

presenting the virtual content on a display of the computing device, the virtual content displayed over a live image of the first physical object that is being concurrently presented on the display of the computing device.

17. The non-transitory computer-readable medium of claim 15 , wherein the computing device is a head-mounted viewing device comprising a transparent display that allows a user wearing the head-mounted viewing device to view content presented on the transparent display while also viewing physical objects visible through the transparent display, the operations further comprising:

presenting the virtual content at a position on the transparent display that overlaps with the user's view of the first physical object.

18. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

receiving, from the remote server, the primary content dataset and a set of virtual content corresponding to the primary content dataset, the primary content dataset including images from the full content dataset that were most commonly captured by a plurality of computing devices; and

storing the primary content dataset and the set of virtual content in a local memory of the computing device.

19. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

capturing an image of a second physical object, the second physical object being different than the first physical object;

comparing the image of the second physical object to images included in the primary content dataset, yielding a second comparison;

determining, based on the second comparison, that the image of the second physical object matches a first image from the primary content dataset; and

presenting, on a display of the computing device, virtual content corresponding to the first image from the primary content dataset.

20. The non-transitory computer-readable medium of claim 15 , wherein comparing the image to images included in the primary content dataset comprises:

identifying a visual reference captured in the image; and

comparing the visual reference captured in the image to visual references captured in the images included in the primary content dataset.

Assignments (12)

Continuity (3)