IP Library Granted Patent US 9,613,462
Granted Patent B2
US 9,613,462 · App. 14/747,043 · Granted Apr 4, 2017

Contextual local image recognition dataset

Inventor: Brian Mullins (Sierra Madre, CA)
Assignee: DAQRI, LLC
G06T19/006G06F1/1686G06F3/005G06F3/147G06F17/30244G06F17/30876G06T1/0007G06F3/011G06T2200/04H04N1/00244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,613,462
App. No.
14/747,043
Granted
Apr 4, 2017
Kind
B2
Abstract

A contextual local image recognition module of a device retrieves a primary content dataset from a server and then generates and updates a contextual content dataset based on an image captured with the device. The device stores the primary content dataset and the contextual content dataset. The primary content dataset comprises a first set of images and corresponding virtual object models. The contextual content dataset comprises a second set of images and corresponding virtual object models retrieved from the server.

Claims (58)

1. A server comprising:

a network communication interface configured to:

communicate with a plurality of devices, and

receive, from a first device from the plurality of devices, an image captured by an optical sensor of the first device;

one or more hardware processors configured to:

generate a primary content dataset comprising a first set of images and corresponding virtual content, the first set of images including a core set of images,

compare the image received from the first device with the first set of images,

based on the comparison, retrieve virtual content associated with the image in response to determining that the first set of images does not include the image received from the first device, and

communicate, via the network communication interface, to the first device, the virtual content associated with the image received from the first device, the first device being configured to update a contextual content dataset with the virtual content associated with the image received from the device; and

a storage device coupled to the one or more hardware processors, the storage device comprising the primary content dataset, a secondary content dataset corresponding to the contextual content dataset of the plurality of devices, and a third content dataset comprising virtual content not included in the primary and secondary content datasets.

2. The server of claim 1 , wherein the core set of images includes images that were most commonly captured by the plurality of devices.

3. The server of claim 1 , wherein the one or more hardware processors are further configured to:

provide the primary content dataset to the plurality of devices;

generate a corresponding secondary content dataset for each device based on images captured from the corresponding device; and

generate a corresponding third content dataset for each device based on usage conditions of the corresponding device.

4. The server of claim 1 , wherein the one or more hardware processors are further configured to:

determine usage conditions of the first device;

identify virtual content based on the usage conditions and not included in the primary and secondary content datasets; and

communicate the virtual content based on the usage conditions and not included in the primary and secondary content datasets to the device.

5. The server of claim 4 , wherein the one or more hardware processors are further configured to:

update the third content dataset of the first device with the virtual content based on the usage conditions and not included in the primary and secondary content datasets.

6. The server of claim 4 , wherein the usage conditions of the first device include at least one of a time, duration, location, and orientation of the device at the time of capture of the image.

7. The server of claim 4 , wherein the usage conditions of the first device include images from other users in a social network of the user of the device.

8. The server of claim 1 , wherein the first device includes an augmented reality application configured to display the virtual content in a display of the device.

9. The server of claim 8 , wherein the one or more hardware processors are further configured to:

upload the primary content dataset to the augmented reality application of the first device in response to detecting an initialization of the augmented reality application in the first device.

10. The server of claim 8 , wherein usage conditions of the first device include at least one of a time and location of the first device while the augmented reality application operates in the first device.

11. A method comprising:

receiving, from a first device from a plurality of devices, an image captured by an optical sensor of the first device;

generating, at a server, a primary content dataset comprising a first set of images and corresponding virtual content, the first set of images including a core set of images;

comparing the image received from the first device with the first set of images;

based on the comparison, retrieving virtual content associated with the image in response to determining that the first set of images does not include the image captured by the first device;

communicating, to the first device, the virtual content associated with the image received from the first device, the first device being configured to update a contextual content dataset with the virtual content associated with the image received from the device; and

storing, at the server, the primary content dataset, a secondary content dataset corresponding to the contextual content dataset of the plurality of devices, and a third content dataset comprising virtual content not included in the primary and secondary content datasets.

12. The method of claim 11 , wherein the core set of images includes images that were most commonly captured from the plurality of devices.

13. The method of claim 11 , further comprising:

providing the primary content dataset to the plurality of devices;

generating a corresponding secondary content dataset for each device based on images captured from the corresponding device; and

generating a corresponding third content dataset for each device based on usage conditions of the corresponding device.

14. The method of claim 11 , further comprising:

determining usage conditions of the first device;

identifying virtual content based on the usage conditions and not included in the primary and secondary content datasets; and

communicating the virtual content based on the usage conditions and not included in the primary and secondary content datasets to the first device.

15. The method of claim 14 , further comprising:

updating the third content dataset of the first device with the virtual content based on the usage conditions and not included in the primary and secondary content datasets.

16. The method of claim 14 , wherein the usage conditions of the first device include at least one of a time, duration, location, and orientation of the first device at the time of capture of the image.

17. The method of claim 14 , wherein the usage conditions of the first device include images from other users in a social network of the user of the first device.

18. The method of claim 11 , wherein the first device includes an augmented reality application configured to display the virtual content in a display of the first device.

19. The method of claim 18 , further comprising:

uploading the primary content dataset to the augmented reality application of the first device in response to detecting an initialization of the augmented reality application in the first device,

wherein usage conditions of the first device include at least one of a time and location of the first device while the augmented reality application operates in the first device.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a server, cause the server to perform operations comprising:

receiving, from a first device from a plurality of devices, an image captured by an optical sensor of the first device;

generating a primary content dataset comprising a first set images and corresponding virtual content, the first set of images including a core set of images;

comparing the image received from the first device with the first set of images;

based on the comparison, retrieving virtual content associated with the image in response to determining that the first set of images does not include the image captured by the first device;

communicating, to the first device, the virtual content associated with the image received from the first device, the first device being configured to update a contextual content dataset with the virtual content associated with the image received from the device; and

storing, at the server, the primary content dataset, a secondary content dataset corresponding to the contextual content dataset of the plurality of devices, and a third content dataset comprising virtual content not included in the primary and secondary content datasets.

Assignments (12)
PATENT SECURITY AGREEMENT Recorded Aug 6, 2024
From: RPX CORPORATION; RPX CLEARINGHOUSE LLC
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 068328/0674 →
RELEASE OF LIEN ON PATENTS Recorded Aug 5, 2024
From: BARINGS FINANCE LLC
To: RPX CORPORATION
Reel/Frame 068328/0278 →
RELEASE OF SECURITY INTEREST Recorded Oct 26, 2020
From: JEFFERIES FINANCE LLC
To: RPX CORPORATION
Reel/Frame 054486/0422 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054244/0566 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054198/0029 →
RELEASE OF SECURITY INTEREST Recorded Aug 14, 2020
From: AR HOLDINGS I, LLC
To: DAQRI, LLC
Reel/Frame 053498/0580 →
PATENT SECURITY AGREEMENT Recorded Aug 14, 2020
From: RPX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 053498/0095 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2020
From: DAQRI, LLC
To: RPX CORPORATION
Reel/Frame 053413/0642 →
RELEASE OF SECURITY INTEREST Recorded Oct 23, 2019
From: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
To: DAQRI, LLC
Reel/Frame 050805/0606 →
LIEN Recorded Oct 8, 2019
From: DAQRI, LLC
To: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
Reel/Frame 050672/0601 →
SECURITY INTEREST Recorded Jun 26, 2019
From: DAQRI, LLC
To: AR HOLDINGS I LLC
Reel/Frame 049596/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2016
From: MULLINS, BRIAN
To: DAQRI, LLC
Reel/Frame 039996/0786 →
Continuity (2)
Continuation 13839818 · Mar 15, 2013
Related Publication 20150371448A1 · Dec 24, 2015