IP Library Granted Patent US 9,412,205
Granted Patent B2
US 9,412,205 · App. 14/467,428 · Granted Aug 9, 2016

Extracting sensor data for augmented reality content

Inventors: Brian Mullins (Sierra Madre, CA); Matthew Kammerait (West Hollywood, CA); Christopher Broaddus (Mountain View, CA)
Assignee: DAQRI, LLC
G06T19/006G02B27/017
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,412,205
App. No.
14/467,428
Granted
Aug 9, 2016
Kind
B2
Abstract

A system and method for extracting data for augmented reality content are described. A device identifies a sensing device using an image captured with at least one camera of the device. Visual data are extracted from the sensing device. The device generates an AR content based on the extracted visual data and maps and displays the AR content in the display to form a layer on the sensing device.

Claims (68)

1. A device comprising:

at least one camera;

a display; and

a hardware processor comprising an augmented reality (AR) application, the AR application configured to include:

an object recognition module to identify a sensing device, an image of which is captured with the at least one camera of the device;

a data extraction module to extract visual data from the sensing device;

an AR content generator to generate AR content based on the extracted visual data; and

an AR content mapping module to map the AR content in the display to form a layer visually perceived on the sensing device.

2. The device of claim 1 , wherein the object recognition module further comprises:

a feature points module to identify feature points of objects in pictures received from the at least one camera, and to recognize the sensing device from the feature points of the sensing device in pictures received from the at least one camera.

3. The device of claim 1 , wherein the data extraction module further comprises:

a units identifier module to identify measuring units associated with the sensing device;

a range identifier module to identify a range of the sensing device; and

a level identifier module to identify a measured level from the sensing device.

4. The device of claim 3 , wherein the AR content generator further comprises:

an AR range identifier module to generate a virtual range indicator for the range of the sensing device;

an AR level identifier module to generate a virtual measured level indicator for the measured level from the sensing device; and

a digital gauge module to generate a virtual digital gauge based on the measured level from the sensing device.

5. The device of claim 4 , wherein the AR content mapping module further comprises:

an AR range mapping module to map a first location in the display to display the virtual range indicator as a layer on the sensing device;

an AR level identifier module to map a second location in the display to display the virtual measured level indicator as a layer on the sensing device; and

a digital gauge module to map a third location in the display to display the virtual digital gauge in the display.

6. The device of claim 3 , wherein the sensing device comprises a gauge with a dial, and further:

the units identifier module to identify measuring units displayed in a center of the dial;

the range identifier module to identify a range of the gauge based on a minimum range level and a maximum range level, both displayed in the dial; and

the level identifier module to identify the measured level from the sensing device based on a dial position relative to the gauge.

7. The device of claim 1 , wherein the AR content generator is to access notification parameters related to the sensing device, and to generate an alert notification in the display in response to a measured level exceeding a threshold of the notification parameters.

8. The device of claim 1 , wherein the AR application is to communicate the image of the sensing device to a server, to receive AR server content from the server, the server to extract visual data from the image of the sensing device, and generate the AR server content based on the extracted visual data, the server not in communication with the sensing device.

9. The device of claim 1 , wherein the AR application is to communicate the image of the sensing device to a server, the server to identify the sensing device from the image, to extract visual data from the image, to access notification parameters associated with the identified sensing device, to generate an alert notification in the display in response to one of the visual data exceeding a threshold of the notification parameters.

10. The device of claim 1 , further comprising:

a geolocation sensor to identify a geographic location of the device, wherein the object recognition module is to identify the sensing device based on the image of the sensing device and the geographic location of the device.

11. A method comprising:

identifying a sensing device using an image captured with at least one camera of a device;

extracting, by a hardware processor, visual data from the sensing device;

generating augmented reality (AR) content based on the extracted visual data; and

mapping and displaying the AR content in a display to form a layer visually perceived on the sensing device.

12. The method of claim 11 , further comprising:

identifying feature points of objects in pictures received from the at least one camera; and

recognizing the sensing device from the features points of the sensing device in pictures received from the at least one camera.

13. The method of claim 11 , further comprising:

identifying measuring units associated with the sensing device;

identifying a range of the sensing device; and

identifying a measured level from the sensing device.

14. The method of claim 13 , further comprising:

generating a virtual range indicator for the range of the sensing device;

generating a virtual measured level indicator for the measured level from the sensing device; and

generating a virtual digital gauge based on the measured level from the sensing device.

15. The method of claim 14 , further comprising:

mapping a first location in the display to display the virtual range indicator as a layer on the sensing device;

mapping a second location in the display to display the virtual measured level indicator as a layer visually perceived on the sensing device; and

mapping a third location in the display to display the virtual digital gauge in the display.

16. The method of claim 14 , further comprising:

accessing notification parameters related to the sensing device; and

generating an alert notification in the display in response to the measured level exceeding a threshold of the notification parameters.

17. The method of claim 11 , further comprising:

communicating the image of the sensing device to a server; and

receiving an AR server content from the server, the server to extract visual data from the image of the sensing device, and generate the AR server content based on the extracted visual data, the server not in communication with the sensing device.

18. The method of claim 11 , further comprising:

communicating the image of the sensing device to a server, the server to identify the sensing device from the image, to extract visual data from the image, to access notification parameters associated with the identified sensing device, to generate an alert notification in the display in response to one of the visual data exceeding a threshold of the notification parameters; and

receiving the alert notification from the server.

19. The method of claim 11 , further comprising:

identifying a geographic location of the device using a geolocation sensor in the device; and

identifying the sensing device based on the image of the sensing device and the geographic location of the device.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

identifying a sensing device using an image captured with at least one camera of a device;

extracting visual data from the sensing device;

generating AR content based on the extracted visual data; and

mapping and displaying the AR content in a display to form a layer visually perceived on the sensing device.

Assignments (12)
CHANGE OF NAME Recorded Aug 3, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060936/0494 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2021
From: RPX CORPORATION
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 056777/0588 →
RELEASE OF SECURITY INTEREST Recorded Oct 26, 2020
From: JEFFERIES FINANCE LLC
To: RPX CORPORATION
Reel/Frame 054486/0422 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054244/0566 →
PATENT SECURITY AGREEMENT Recorded Oct 23, 2020
From: RPX CLEARINGHOUSE LLC; RPX CORPORATION
To: BARINGS FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 054198/0029 →
PATENT SECURITY AGREEMENT Recorded Aug 14, 2020
From: RPX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 053498/0095 →
RELEASE OF SECURITY INTEREST Recorded Aug 14, 2020
From: AR HOLDINGS I, LLC
To: DAQRI, LLC
Reel/Frame 053498/0580 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2020
From: DAQRI, LLC
To: RPX CORPORATION
Reel/Frame 053413/0642 →
RELEASE OF SECURITY INTEREST Recorded Oct 23, 2019
From: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
To: DAQRI, LLC
Reel/Frame 050805/0606 →
LIEN Recorded Oct 8, 2019
From: DAQRI, LLC
To: SCHWEGMAN, LUNDBERG & WOESSNER, P.A.
Reel/Frame 050672/0601 →
SECURITY INTEREST Recorded Jun 26, 2019
From: DAQRI, LLC
To: AR HOLDINGS I LLC
Reel/Frame 049596/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2015
From: MULLINS, BRIAN; KAMMERAIT, MATTHEW; BROADDUS, CHRISTOPHER
To: DAQRI, LLC
Reel/Frame 035171/0063 →
Continuity (1)
Related Publication 20160055674A1 · Feb 25, 2016