IP Library Granted Patent US 10,504,294
Granted Patent B2
US 10,504,294 · App. 16/199,780 · Granted Dec 10, 2019

System and method for augmented reality discrepancy determination and reporting

Inventors: Brian Bare (Chesapeake, VA); Jonathan Martin (Virginia Beach, VA); Patrick Ryan (Newport News, VA); Mark Lawrence (Williamsburg, VA); Paul Sells (Oxnard, CA)
Assignee: Huntington Ingalls Incorporated
G06T19/006G06F3/04815G06K9/00664G06K9/00671G06T15/205H04W4/021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,504,294
App. No.
16/199,780
Granted
Dec 10, 2019
Kind
B2
Abstract

A method is provided for providing target object information to a mobile interface device user in a dynamic structural environment. The method includes receiving from the mobile interface device a request for target object information associated with a target object in the dynamic structural environment. A pose of the mobile interface device relative to the target object is determined accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure. The method also includes assembling AR target object information for transmission to and display on the mobile interface device and transmitting the AR target object information to the mobile interface device for display in conjunction with a real-time view of the target object.

Claims (46)

1. A method for providing target object information to a mobile interface device user in a dynamic structural environment in the form of a dynamic structure having an environment coordinate system, the method comprising:

receiving on a central data processor from the mobile interface device over a communication network a request for target object information associated with a target object in the dynamic structural environment;

determining a pose of the mobile interface device relative to the target object accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure;

assembling augmented reality (AR) target object information for transmission to and display on the mobile interface device; and

transmitting the AR target object information to the mobile interface device for display in conjunction with a real-time view of the target object.

2. The method of claim 1 further comprising:

receiving first spatial information from a local positioning system and second spatial information from a three dimensional model of the dynamic structure; and

using the first and second spatial information to determine the spatial differences in the environment coordinate system resulting from changes in the dynamic structure.

3. The method of claim 1 further comprising:

receiving a digital image of the target object captured by the mobile interface device; and

receiving mobile interface device location information from a local positioning system, said location information being provided in the environment coordinate system,

wherein the location information and the digital image are used in the action of determining the pose of the mobile interface device.

4. The method of claim 1 wherein the AR target object information includes at least one of the set consisting of information on a desired target object parameter, and

a graphical representation of a desired target object configuration.

5. The method of claim 1 further comprising:

receiving, from a measurement data system, target object data for one or more target object parameters.

6. The method of claim 5 wherein the AR target object information includes a graphical representation of the one or more target object parameters, the graphical representation being constructed using the target object data.

7. The method of claim 5 further comprising:

comparing the target object data to desired target object values for the one or more target object parameters to determine target object parameter deviations.

8. The method of claim 7 wherein the AR target object information includes a graphical representation of the target object parameter deviations.

9. The method of claim 7 further comprising:

performing at least one of the set consisting of a maintenance operation and a discrepancy correction operation on the target object; and

repeating the actions of receiving target object data, comparing the target object data to desired target object values to determine target object parameter deviations, assembling AR target object information, and transmitting the AR target information to the mobile interface device for display.

10. The method of claim 1 , wherein the mobile interface device is one of the set consisting of a tablet computer, a smartphone, and a wearable heads-up display.

11. The method of claim 1 wherein the request for target environment information includes a digital image of the target object captured by the mobile interface device.

12. The method of claim 1 wherein the dynamic structure defines an interior space in which the target object is disposed.

13. The method of claim 1 wherein the dynamic structure comprises the target object.

14. The method of claim 1 wherein the spatial changes result from flexure of the dynamic structure.

15. An automated system for providing target object information to a mobile interface device for use in inspecting a target object in a dynamic structural environment in the form of a dynamic structure having an environment coordinate system, the system comprising:

at least one mobile interface device configured for variable disposition within the dynamic structural environment, capturing information about the target object within the structural environment, and transmitting the target object information over a network;

a local positioning system in communication with the at least one mobile interface device via the network; and

a central processor comprising at least one data processing machine in communication with the at least one mobile interface device and the local positioning system via the network, the central processor being configured for

receiving from a requesting one of the at least one mobile interface device a request for target object information associated with the target object;

determining a pose of the requesting mobile interface device relative to the target object accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure;

assembling augmented reality (AR) target object information for transmission to and display on the mobile interface device; and

transmitting the AR target object information to the mobile interface device for display in conjunction with a real-time view of the target object.

16. The automated system of claim 15 wherein the central processor is also configured for

receiving first spatial information from a local positioning system and second spatial information from a three dimensional model of the dynamic structure; and

using the first and second spatial information to determine the spatial differences in the environment coordinate system resulting from changes in the dynamic structure.

17. The automated system of claim 15 further comprising:

a measurement data system configured for capturing target object data for one or more target object parameters.

18. The automated system of claim 17 ,

wherein the central processor is also configured for

receiving the target object data, and

comparing the target object data to desired target object values for the one or more target object parameters to determine target object parameter deviations, and

wherein the AR target object information includes a graphical representation of the target object parameter deviations.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 31, 2019
From: SELLS, PAUL
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 048210/0827 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: BARE, BRIAN
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 047581/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: MARTIN, JONATHAN
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 047581/0740 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: RYAN, PATRICK
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 047581/0840 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: LAWRENCE, MARK
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 047581/0923 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2018
From: SELLS, PAUL
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 047582/0068 →
Continuity (5)
Continuation In Part 14729561 · Jun 3, 2015
Provisional Application 62036762 · Aug 13, 2014
Provisional Application 62025041 · Jul 16, 2014
Provisional Application 61997793 · Jun 9, 2014
Related Publication 20190096138A1 · Mar 28, 2019