IP Library Granted Patent US 11,062,517
Granted Patent B2
US 11,062,517 · App. 16/036,384 · Granted Jul 13, 2021

Virtual access to a limited-access object

Inventors: James Aaron Crews (Cedar Park, TX); Trevor Duncan Schleiss (Austin, TX); Benjamin Michael Bishop (Orlando, FL)
Assignee: FISHER-ROSEMOUNT SYSTEMS, INC.
G06T19/006G06F1/163G06F3/011G06F3/012G06F3/167
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,062,517
App. No.
16/036,384
Granted
Jul 13, 2021
Kind
B2
Abstract

In a method of facilitating interaction between a user of an augmented reality (AR) mobile device and a first real-world object, a display device is caused to superimpose digital information on portions of a process control environment within a field of view of a camera of the device. The superimposed information is associated with nodes in a map of the environment, and the nodes correspond to other objects in the environment. The display is caused to indicate a direction to the first object. After detecting a user input that indicates selection of the first object, the display is caused to superimpose, on a portion of the process control environment currently within the field of view, a digital model or image of the first object. A user interface is caused to provide one or more virtual controls and/or one or more displays associated with the first object.

Claims (42)

1. A method of facilitating interaction between a user of an augmented reality (AR) mobile device and a first real-world object in a process control environment, the method comprising:

causing, by one or more processors of an AR system and as the user moves through the process control environment, a display of the AR mobile device to superimpose digital information on portions of the process control environment that are within a field of view of a camera of the AR mobile device, the superimposed information being associated with nodes in a map of the process control environment, and the nodes corresponding to other real-world objects in the process control environment;

causing, by the one or more processors, the display of the AR mobile device to indicate a direction to the first real-world object, while the first real-world object is not located within the field of view of the camera; and

after detecting a user input, made via the AR mobile device, that indicates selection of the first real-world object, causing, by the one or more processors and while the first real-world object is not located within the field of view of the camera, the display of the AR mobile device to superimpose, on a portion of the process control environment that is currently within the field of view of the camera but does not include the first real-world object,

a digital model or digital image of the first real-world object, and

a user interface providing at least (i) one or more virtual controls providing user control of the first real-world object, or (ii) one or more displays presenting information associated with settings or outputs of the first real-world object.

2. The method of claim 1 , wherein causing the display of the AR mobile device to indicate the direction to the first real-world object occurs after detecting the user input that indicates selection of the first real-world object.

3. The method of claim 2 , wherein the user input that indicates selection of the first real-world object includes a voice command requesting a user interface for the first real-world object.

4. The method of claim 2 , wherein causing the display of the AR mobile device to indicate the direction to the first real-world object includes:

causing an arrow pointing in the direction of the first real-world object to be superimposed, along with the digital model or digital image, on the portion of the process control environment.

5. The method of claim 1 , wherein causing the display of the AR mobile device to indicate the direction to the first real-world object occurs before detecting the user input that indicates selection of the first real-world object.

6. The method of claim 5 , wherein causing the display of the AR mobile device to indicate the direction to the first real-world object occurs in response to detecting (i) that the first real-world object is within a threshold distance of the AR mobile device and (ii) that the first real-world object is not currently within the field of view of the camera.

7. The method of claim 1 , wherein causing the display of the AR mobile device to superimpose, on the portion of the process control environment, the digital model or digital image includes:

using a web services interface to retrieve the digital model or digital image from a database.

8. The method of claim 1 , wherein causing the display of the AR mobile device to superimpose, on the portion of the process control environment, the digital model or digital image and the user interface includes:

in response to detecting a user selection of the digital model or digital image, causing the display to also superimpose the user interface on the portion of the process control environment.

9. A server for facilitating interaction between a user of an augmented reality (AR) mobile device and a first real-world object in a process control environment, the server being configured to:

cause, as the user moves through the process control environment, a display of the AR mobile device to superimpose digital information on portions of the process control environment that are within a field of view of a camera of the AR mobile device, the superimposed information being associated with nodes in a map of the process control environment, and the nodes corresponding to other real-world objects in the process control environment;

cause the display of the AR mobile device to indicate a direction to the first real-world object while the first real-world object is not located within the field of view of the camera; and

after detecting a user input, made via the AR mobile device, that indicates selection of the first real-world object, and while the first real-world object is not located within the field of view of the camera, causing the display of the AR mobile device to superimpose, on a portion of the process control environment that is currently within the field of view of the camera but does not include the first real-world object,

a digital model or digital image of the first real-world object, and

a user interface providing at least (i) one or more virtual controls providing user control of the first real-world object, or (ii) one or more displays presenting information associated with settings or outputs of the first real-world object.

10. The server of claim 9 , wherein the server is configured to cause the display of the AR mobile device to indicate the direction to the first real-world object after detecting the user input that indicates selection of the first real-world object.

11. The server of claim 10 , wherein the user input that indicates selection of the first real-world object includes a voice command requesting a user interface for the first real-world object.

12. The server of claim 10 , wherein the server is configured to cause the display of the AR mobile device to indicate the direction to the first real-world object at least by:

causing an arrow pointing in the direction of the first real-world object to be superimposed, along with the digital model or digital image, on the portion of the process control environment.

13. The server of claim 9 , wherein the server is configured to cause the display of the AR mobile device to indicate the direction to the first real-world object before detecting the user input that indicates selection of the first real-world object.

14. The server of claim 13 , wherein the server is configured to cause the display of the AR mobile device to indicate the direction to the first real-world object in response to detecting (i) that the first real-world object is within a threshold distance of the AR mobile device and (ii) that the first real-world object is not currently within the field of view of the camera.

15. The server of claim 9 , wherein the server is configured to cause the display of the AR mobile device to superimpose, on the portion of the process control environment, the digital model or digital image at least by:

using a web services interface to retrieve the digital model or digital image from a database.

16. The server of claim 9 , wherein the server is configured to cause the display of the AR mobile device to superimpose, on the portion of the process control environment, the digital model or digital image and the user interface at least by:

in response to detecting a user selection of the digital model or digital image, causing the display to also superimpose the user interface on the portion of the process control environment.

17. An augmented reality (AR) mobile device for facilitating interaction between a user and a first real-world object in a process control environment, the AR mobile device including a camera and a display, and the AR mobile device being configured to:

superimpose, as the user moves through the process control environment and via the display, digital information on portions of the process control environment that are within a field of view of the camera, the superimposed information being associated with nodes in a map of the process control environment, and the nodes corresponding to other real-world objects in the process control environment;

indicate, via the display, a direction to the first real-world object while the first real-world object is not located within the field of view of the camera; and

after detecting a user input, made via the AR mobile device, that indicates selection of the first real-world object, and while the first real-world object is not located within the field of view of the camera, superimpose, on a portion of the process control environment that is currently within the field of view of the camera but does not include the first real-world object, and via the display,

a digital model or digital image of the first real-world object, and

a user interface providing at least (i) one or more virtual controls providing user control of the first real-world object, or (ii) one or more displays presenting information associated with settings or outputs of the first real-world object.

18. The AR mobile device of claim 17 , wherein the AR mobile device is configured to indicate, via the display, the direction to the first real-world object after detecting the user input that indicates selection of the first real-world object.

19. The AR mobile device of claim 17 , wherein the AR mobile device is configured to indicate, via the display, the direction to the first real-world object at least by:

causing an arrow pointing in the direction of the first real-world object to be superimposed, along with the digital model or digital image, on the portion of the process control environment.

20. The AR mobile device of claim 17 , wherein the AR mobile device is configured to indicate the direction to the first real-world object in response to detecting (i) that the first real-world object is within a threshold distance of the AR mobile device and (ii) that the first real-world object is not currently within the field of view of the camera.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 18, 2018
From: CREWS, JAMES AARON; SCHLEISS, TREVOR DUNCAN; BISHOP, BENJAMIN MICHAEL
To: FISHER-ROSEMOUNT SYSTEMS, INC.
Reel/Frame 046382/0597 →
Continuity (2)
Provisional Application 62564074 · Sep 27, 2017
Related Publication 20190096133A1 · Mar 28, 2019
Cited By (7)
US 12,267,482 US 12,282,604 US 12,322,052 US 12,417,593 US 12,449,891 US 12,519,924 US 12,718,494