IP Library Granted Patent US 10,360,728
Granted Patent B2
US 10,360,728 · App. 14/715,672 · Granted Jul 23, 2019

Augmented reality device, system, and method for safety

Inventors: Sharath Venkatesha (Minnetonka, MN); Erik Todeschini (Camillus, NY); Donald Anderson (Locke, NY)
Assignee: HAND HELD PRODUCTS, INC.
G06T19/006G08B7/06G08B21/14G01S5/02G01S19/17G05B2219/32014G06T2207/30108G08B7/066
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,360,728
App. No.
14/715,672
Granted
Jul 23, 2019
Kind
B2
Abstract

Disclosed is a device, system, and method utilizing augmented-reality (AR) for safety. The AR safety system generates hazard-response messages in response to sensor data in order to help users understand and response to hazards in work environments in an intuitive and effective way. The AR safety system fuses a plurality of technologies. These technologies include person location/tracking, hazard detection/tracking, and augmented reality (i.e., AR).

Claims (51)

1. An augmented reality (AR) hazard display, comprising:

a user tracker tracking a user's position in real-time;

an audio source to produce sounds and/or audio prompts for the user;

a communicator receiving real-time data indicative of a hazard from a plurality of sensors, the plurality of sensors comprising:

a gas sensor for detecting a gas and providing gas data, the gas sensor being positioned remotely from the augmented reality hazard display, the gas sensor being in a fixed geographical position; and

a weather sensor for providing wind data;

a processor configured to:

in response to the hazard, create a geo-fence indicative of virtual barriers surrounding a hazardous area of the hazard based on receiving the real-time data;

combine the user's position tracked by the user tracker and the real-time data to provide an augmented reality (AR) message when the user enters the geo-fence, wherein the augmented reality (AR) message comprises two or more of a hazard identification indicative of an extent and/or severity of the hazard, a map of an area contaminated by hazard, a direction of hazard travel, and an estimation of future hazard movement;

in response to the hazard, estimating movement of the gas based on the gas data and the wind data;

combine the user's position tracked by the user tracker and the estimated movement of the gas to provide an augmented reality (AR) message comprising visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via the audio source; and

an interface to convey the AR messages to the user.

2. The augmented reality (AR) hazard display according to claim 1 , wherein the user tracker comprises a global positioning system (GPS) receiver.

3. The augmented reality (AR) hazard display according to claim 1 , wherein the user tracker comprises at least one camera.

4. The augmented reality (AR) hazard display according to claim 1 , wherein the user tracker comprises a wireless-network receiver.

5. The augmented reality (AR) hazard display according to claim 1 , wherein the audio sources are configured to create three-dimensional (3D) audio.

6. The augmented reality (AR) hazard display according to claim 1 , wherein the interface comprises a head mounted display (HMD).

7. The augmented reality (AR) hazard display according to claim 1 , wherein the visual images of the map show the user's position and a route to a safe place.

8. The augmented reality (AR) hazard display according to claim 1 , wherein the augmented reality (AR) message provided when the user enters the geo-fence comprises prompts to guide a user's movement.

9. An augmented reality (AR) safety system, comprising:

a hazard tracker comprising:

a gas sensor for detecting a gas and providing gas data, the gas sensor being positioned remotely from an AR hazard display; and

a weather sensor for providing wind data;

an audio source to produce sounds and/or audio prompts;

a user tracker for tracking a user's position; and

the AR hazard display communicatively coupled to the hazard tracker, the audio source, and the user tracker, the AR hazard display adapted to:

in response to detected gas:

estimate movement of the gas based on the gas data and the wind data; and

create a geo-fence indicative of virtual barriers surrounding a hazardous area based on the estimated movement of the gas;

combine the tracked user-position and estimated movement of the gas to generate AR messages;

present the AR messages to the user when the user enters the geo-fence, the AR messages comprising two or more of hazard identity indicative of an extent and/or severity of the detected gas, a map of an area contaminated by the gas, a direction of gas travel, and an estimation of future gas movement;

present visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place based on the tracked user-position and the estimated movement of the gas; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via the audio source.

10. The augmented reality (AR) safety system according to claim 9 , wherein the AR hazard display is a smartphone.

11. The augmented reality (AR) safety system according to claim 9 , wherein the user tracker and/or the AR hazard display is worn by the user.

12. The augmented reality (AR) safety system according to claim 9 , wherein the gas sensor detects an invisible gas.

13. The augmented reality (AR) safety system according to claim 9 , wherein the AR messages comprise graphics to help a user visualize the gas.

14. The augmented reality (AR) safety system according to claim 9 , wherein the AR messages comprise graphics to help a user visualize the direction in which the gas is moving.

15. The augmented reality (AR) safety system according to claim 9 , wherein the AR messages comprise an alert that the user is near a hazardous area or has entered into a hazardous area.

16. The augmented reality (AR) safety system according to claim 9 , wherein the AR messages comprise navigational prompts to guide a user to a safe area.

17. The augmented reality (AR) safety system according to claim 9 , wherein the user tracker comprises at least one camera.

18. A computer-implemented method for generating an augmented-reality (AR) message, the method comprising:

gathering, in real-time, wind data from a weather sensor and gas data from a gas sensor, the gas sensor being in a fixed geographical position;

gathering, in real-time, user-position data from at least one user-tracking sensor;

in response to detected gas;

estimating movement of the gas based on the gas data and the wind data;

creating a geo-fence indicative of virtual barriers surrounding a hazardous area based on the estimated movement of the gas;

generating, based on the gathered user-position data and the gas data, an AR message when the user enters the geo-fence, the AR message comprising two or more of a hazard identity indicative of an extent and/or severity of the detected gas, a map of an area contaminated by the gas, a direction of gas travel, and an estimation of future gas movement;

presenting visual images of a map of an area contaminated by the gas and navigational messages directing the user to a safe place based on the user-position data and the estimated movement of the gas; and

if the user is in a low or no visibility environment, provide the navigational messages directing the user to the safe place via an audio source.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2015
From: VENKATESHA, SHARATH; TODESCHINI, ERIK; ANDERSON, DONALD
To: HAND HELD PRODUCTS, INC.
Reel/Frame 035665/0600 →
Continuity (1)
Related Publication 20160343163A1 · Nov 24, 2016
Cited By (5)
US 12,260,841 US 12,293,319 US 12,353,787 US 12,400,678 US 12,525,112