IP Library Granted Patent US 9,934,754
Granted Patent B2
US 9,934,754 · App. 14/870,701 · Granted Apr 3, 2018

Dynamic sensor array for augmented reality system

Inventors: Brian Mullins (Sierra Madre, CA); Matthew Kammerait (West Hollywood, CA)
Assignee: DAQRI, LLC
G09G5/003G02B27/017G06T19/006G02B27/0093G02B2027/014G02B2027/0178G02B2027/0187G06F3/14G09G3/001G09G2370/022G09G2370/16H04N13/0203H04N13/044H04N13/0484H04N2013/0074
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,934,754
App. No.
14/870,701
Granted
Apr 3, 2018
Kind
B2
Abstract

A system and method for generating a dynamic sensor array for an augmented reality system is described. A head mounted device includes one or more sensors, an augmented reality (AR) application, and a sensor array module. The sensor array module identifies available sensors from other head mounted devices that are geographically located within a predefined area. A dynamic sensor array is formed based on the available sensors and the one or more sensors. The dynamic sensor array is updated based on an operational status of the available sensors and the one or more sensors. The AR application generates AR content based on data from the dynamic sensor array. A display of the head mounted device displays the AR content.

Claims (81)

1. A head mounted device comprising:

one or more sensors;

one or more processors comprising an augmented reality (AR) application and a sensor array module;

the sensor array module configured to:

identify a sensor, in the one or more sensors of the head mounted device, as being a defective sensor;

in response to the identifying the defective sensor, identify sensors included by a plurality of other head mounted devices that are geographically located within a predefined area and that are operational;

form a dynamic sensor array that includes the identified sensors of the plurality of other head mounted devices and includes the one or more sensors of the head mounted device;

identify an operating sensor, in the dynamic sensor array, that is configured to perform a function similar to the defective sensor;

update the dynamic sensor array by excluding the defective sensor from the dynamic sensor array;

the AR application configured to generate AR content based on data from the dynamic sensor array; and

a display configured to display the AR content.

2. The head mounted device of claim 1 , wherein the sensor array module is configured to:

identify an additional sensor from the dynamic sensor array, the additional sensor including a sensor of the plurality of other head mounted devices, the additional sensor performing a function similar to the one or more sensors in the head mounted device, and

access data from the additional sensor,

wherein the AR application is configured to generate the AR content based on data from the additional sensor and the one or more sensors of the head mounted device.

3. The head mounted device of claim 1 , wherein the sensor array module is configured to:

identify a complimentary sensor from the dynamic sensor array, the complimentary sensor including a sensor of the plurality of other head mounted devices, the complimentary sensor performing a function different from the one or more sensors in the head mounted device, and

access data from the complimentary sensor,

wherein the AR application is configured to generate the AR content based on data from the complimentary sensor and the one or more sensors of the head mounted device.

4. The head mounted device of claim 1 , wherein the sensor array module is configured to:

identify a context of the head mounted device,

identify a context sensor from the dynamic sensor array, the context sensor including a sensor of the plurality of other head mounted devices, the context sensor identified based on the context of the head mounted device, and

access data from the context sensor,

wherein the AR application is configured to generate the AR content based on data from the context sensor and the one or more sensors of the head mounted device.

5. The head mounted device of claim 4 , wherein the context include a predefined geographic location,

wherein the AR application is configured to generate the AR content in response to detecting that the head mounted device is located or approaching the predefined geographic location.

6. The head mounted device of claim 4 , wherein the context include a task of the AR application,

wherein the AR application is configured to generate the AR content in response to detecting a selection of the task in the head mounted device.

7. The head mounted device of claim 4 , wherein the context include a user privilege,

wherein the AR application is configured to generate the AR content based on the user privilege.

8. The head mounted device of claim 1 , wherein the sensor array module is configured to identify a plurality of stationary sensors located within the predefined area, the plurality of stationary sensors being external to any head mounted device, and to form the dynamic sensor array based on the plurality of stationary sensors, the available sensors from the plurality of other head mounted devices, and the one or more sensors of the head mounted device.

9. The head mounted device of claim 1 , wherein the sensor array module is configured to identify one or more sensors from the dynamic sensor array based on a physical object captured with a camera of the head mounted device, the dynamic array including a plurality of stationary sensors being external to any head mounted device, and to access data from the identified one or more sensors from the dynamic sensor array,

wherein the AR application is configured to generate the AR content based on the data from the identified one or more sensors from the dynamic sensor array.

10. A server comprising:

one or more processors comprising a sensor array module;

the sensor array module configured to:

identify a sensor, in the one or more sensors of the head mounted device, as being a defective sensor;

in response to the identifying the defective sensor, identify sensors included by a plurality of other head mounted devices that are geographically located within a predefined area and that are operational;

form a dynamic sensor array that includes the identified sensors of the plurality of other head mounted devices and includes the one or more sensors of the head mounted device;

identify an operating sensor, in the dynamic sensor array, that is configured to perform a function similar to the defective sensor;

update the dynamic sensor array by excluding the defective sensor from the dynamic sensor array;

provide data from the dynamic sensor array to a first head mounted device, the first head mounted device configured to generate AR content based on the data from the dynamic sensor array.

11. The server of claim 10 , wherein the sensor array module is configured to:

identify an additional sensor from the dynamic sensor array, the additional sensor including a sensor of the plurality of head mounted devices, the additional sensor performing a function similar to a sensor in the first head mounted device, and

access and provide data from the additional sensor to the first head mounted device.

12. The server of claim 10 , wherein the sensor array module is configured to:

identify a complimentary sensor from the dynamic sensor array, the complimentary sensor including a sensor of the plurality of head mounted devices, the complimentary sensor performing a function different from a sensor in the first head mounted device, and

access and provide data from the complimentary sensor to the first head mounted device.

13. The server of claim 10 , wherein the sensor array module is configured to:

identify a context of the first head mounted device,

identify a context sensor from the dynamic sensor array, the context sensor including a sensor of the plurality of head mounted devices, the context sensor identified based on the context of the first head mounted device, and

access and provide data from the context sensor to the first head mounted device.

14. A computer-implemented method comprising:

identifying a plurality of head mounted devices geographically located within a predefined area;

identifying sensors included by the plurality of head mounted devices;

forming, using a hardware processor of a computer, a dynamic sensor array that includes the identified sensors;

identifying a non-operating sensor and an operating sensor in the dynamic sensor array;

determining that the operating sensor and the non-operating sensor each include a same sensor that is configured to generate a same type of data;

updating the dynamic sensor array by excluding the non-operating sensor from the dynamic sensor array in response to the determining; and

providing data from the dynamic sensor array to a first head mounted device, the first head mounted device configured to generate AR content based on the data from the dynamic sensor array.

15. The computer-implemented method of claim 14 , further comprising:

identifying a defective sensor in the first head mounted device;

identifying a replacement sensor from the dynamic sensor array, the replacement sensor including a sensor of the plurality of head mounted devices, the replacement sensor performing a function similar to the defective sensor; and

accessing and providing data from the replacement sensor to the first head mounted device.

16. The computer-implemented method of claim 14 , further comprising:

identifying an additional sensor from the dynamic sensor array, the additional sensor including a sensor of the plurality of head mounted devices, the additional sensor performing a function similar to a sensor in the first head mounted device; and

accessing and providing data from the additional sensor to the first head mounted device.

17. The computer-implemented method of claim 14 , further comprising:

identifying a complimentary sensor from the dynamic sensor array, the complimentary sensor including a sensor of the plurality of head mounted devices, the complimentary sensor performing a function different from a sensor in the first head mounted device; and

accessing and providing data from the complimentary sensor to the first head mounted device.

18. The computer-implemented method of claim 14 , further comprising:

identifying a context of the first head mounted device;

identifying a context sensor from the dynamic sensor array, the context sensor including a sensor of the plurality of head mounted devices, the context sensor identified based on the context of the first head mounted device; and

accessing and providing data from the context sensor to the first head mounted device.

19. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

identifying a sensor, in one or more sensors of a head mounted device, as being a defective sensor;

in response to the identifying the defective sensor, identifying sensors included by a plurality of other head mounted devices that are geographically located within a predefined area and that are operational;

forming a dynamic sensor array that includes the identified sensors of the plurality of other head mounted devices and includes the one or more sensors of the head mounted device;

identifying an operating sensor, in the dynamic sensor array, that is configured to perform a function similar to the defective sensor;

updating the dynamic sensor array by excluding the defective sensor from the dynamic sensor array;

providing data from the dynamic sensor array to a first head mounted device, the first head mounted device configured to generate AR content based on the data from the dynamic sensor array.

Assignments (12)
PATENT SECURITY AGREEMENT Recorded Aug 6, 2024
From: RPX CORPORATION; RPX CLEARINGHOUSE LLC
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 068328/0674 →
RELEASE OF LIEN ON PATENTS Recorded Aug 5, 2024
From: BARINGS FINANCE LLC
To: RPX CORPORATION
Reel/Frame 068328/0278 →
RELEASE OF SECURITY INTEREST Recorded Oct 26, 2020
From: JEFFERIES FINANCE LLC
To: RPX CORPORATION
Reel/Frame 054486/0422 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054244/0566 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054198/0029 →
RELEASE OF SECURITY INTEREST Recorded Aug 14, 2020
From: AR HOLDINGS I, LLC
To: DAQRI, LLC
Reel/Frame 053498/0580 →
PATENT SECURITY AGREEMENT Recorded Aug 14, 2020
From: RPX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 053498/0095 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2020
From: DAQRI, LLC
To: RPX CORPORATION
Reel/Frame 053413/0642 →
RELEASE OF SECURITY INTEREST Recorded Oct 23, 2019
From: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
To: DAQRI, LLC
Reel/Frame 050805/0606 →
LIEN Recorded Oct 8, 2019
From: DAQRI, LLC
To: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
Reel/Frame 050672/0601 →
SECURITY INTEREST Recorded Jun 26, 2019
From: DAQRI, LLC
To: AR HOLDINGS I LLC
Reel/Frame 049596/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2016
From: MULLINS, BRIAN; KAMMERAIT, MATTHEW
To: DAQRI, LLC
Reel/Frame 038813/0318 →
Continuity (1)
Related Publication 20170092220A1 · Mar 30, 2017