IP Library Granted Patent US 10,915,754
Granted Patent B2
US 10,915,754 · App. 16/657,175 · Granted Feb 9, 2021

System and method for use of augmented reality in outfitting a dynamic structural space

Inventors: Brian Bare (Chesapeake, VA); Jonathan Martin (Virginia Beach, VA); Patrick Ryan (Newport News, VA); Mark Lawrence (Williamsburg, VA); Paul Sells (Oxnard, CA)
Assignee: Huntington Ingalls Incorporated
G06K9/00671G06T7/70H04W4/029G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,915,754
App. No.
16/657,175
Granted
Feb 9, 2021
Kind
B2
Abstract

A method is provided for providing target object information to a mobile interface device user in a dynamic structural environment. The method includes receiving from the mobile interface device a request for target object information associated with a target object in the dynamic structural environment. A pose of the mobile interface device relative to the target object is determined accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure. The method also includes assembling AR target object information for transmission to and display on the mobile interface device and transmitting the AR target object information to the mobile interface device for display in conjunction with a real-time view of the target object.

Claims (60)

1. A method for providing target space information to a mobile interface device user positioned within a dynamic structure having a structure coordinate system, the method comprising:

receiving on a central data processor from the mobile interface device over a communication network a request for target space information associated with a target space within the dynamic structure;

determining a pose of the mobile interface device relative to the target space accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure;

assembling, by the central data processor, augmented reality (AR) target space information for transmission to and display on the mobile interface device; and

transmitting, by the central data processor, the AR target space information to the mobile interface device for display in conjunction with a real-time view of the target space.

2. A method according to claim 1 further comprising:

receiving first spatial information from a local positioning system and second spatial information from a three dimensional model of the dynamic structure; and

using the first and second spatial information to determine the spatial differences in the environment coordinate system resulting from changes in the dynamic structure.

3. A method according to claim 1 further comprising:

receiving a digital image of the target space captured by the mobile interface device; and

receiving mobile interface device location information from a local positioning system, said location information being provided in the environment coordinate system,

wherein the location information and the digital image are used in the action of determining the pose of the mobile interface device.

4. A method according to claim 1 further comprising:

receiving, from a measurement data system, target spatial measurement data for the target space;

determining a current spatial configuration for the target space.

5. A method according to claim 4 further comprising:

retrieving from data storage spatial measurement data information for a previously determined spatial configuration for the target space;

comparing the current spatial configuration to the previously determined spatial configuration for the target space to determine if changes have occurred; and

responsive to a determination that changes have occurred, placing in data storage information on the current spatial configuration.

6. A method according to claim 1 further comprising:

retrieving from data storage object information on one or more objects disposed within the target space,

wherein at least a portion of the object information is included in the AR target space information.

7. A method according to claim 1 further comprising wherein the AR target space information includes information on placement of an object within the target space.

8. A method according to claim 7 wherein the object placement information includes a three dimensional image of the object in a proposed location within the target space, the image being viewable in conjunction with the real-time view of the target space.

9. A method according to claim 7 wherein the object placement information includes graphical representation of a sequence of actions for positioning the object in a desired location within the target space.

10. A method according to claim 8 wherein the object placement information includes information on interferences between the object and the dynamic structure or other objects within the target space when the object is in the proposed location.

11. A method according to claim 1 , wherein the mobile interface device is one of the set consisting of a tablet computer, a smartphone, and a wearable heads-up display.

12. A method according to claim 1 wherein the request for target environment information includes a digital image of the target space captured by the mobile interface device.

13. A method according to claim 1 wherein the spatial changes result from flexure of the dynamic structure.

14. An automated system for providing to a mobile interface device target space information on a target space within a dynamic structure having an environment coordinate system, the system comprising:

at least one mobile interface device configured for variable disposition within the dynamic structure, capturing information about the target space, and transmitting the target space information over a network;

a local positioning system in communication with the at least one mobile interface device via the network; and

a central data processor comprising at least one data processing machine in communication with the at least one mobile interface device and the local positioning system via the network, the central data processor being configured for

receiving from a requesting one of the at least one mobile interface device a request for target space information;

determining a pose of the requesting mobile interface device relative to the target space accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure;

assembling augmented reality (AR) target space information for transmission to and display on the mobile interface device; and

transmitting the AR target object information to the mobile interface device for display in conjunction with a real-time view of the target space.

15. A system according to claim 14 wherein the central data processor is also configured for

receiving first spatial information from the local positioning system and second spatial information from a three dimensional model of the dynamic structure; and

using the first and second spatial information to determine the spatial differences in the environment coordinate system resulting from changes in the dynamic structure.

16. A system according to claim 14 further comprising:

a measurement data system configured for capturing target space spatial measurement data for one or more target space parameters.

17. A system according to claim 16 ,

wherein the central data processor is also configured for

receiving the target spatial measurement data from the measurement data system; and

determining a current spatial configuration for the target space.

18. A system according to claim 17 ,

wherein the central data processor is also configured for

comparing the target object data to desired target object values for the one or more target object parameters to determine target object parameter deviations, and

responsive to a determination that changes have occurred, placing in data storage information on the current spatial configuration.

19. A method for providing target space information to a mobile interface device user positioned within a dynamic structure having a structure coordinate system, the method comprising:

receiving on a central data processor from the mobile interface device over a communication network a request for target space information associated with a target space within the dynamic structure;

determining a pose of the mobile interface device relative to the target space accounting for spatial differences in the environment coordinate system resulting from changes in the dynamic structure;

receiving, from a measurement data system, target spatial measurement data for the target space;

determining a current spatial configuration for the target space;

retrieving from data storage spatial measurement data information for a previously determined spatial configuration for the target space;

comparing the current spatial configuration to the previously determined spatial configuration for the target space to determine if changes have occurred; and

responsive to a determination that changes have occurred, placing in data storage information on the current spatial configuration;

assembling, by the central data processor, augmented reality (AR) target space information for transmission to and display on the mobile interface device; and

transmitting, by the central data processor, the AR target space information to the mobile interface device for display in conjunction with a real-time view of the target space.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: BARE, BRIAN
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 050760/0663 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: MARTIN, JONATHAN
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 050760/0731 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: RYAN, PATRICK
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 050760/0932 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: LAWRENCE, MARK
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 050760/0994 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: SELLS, PAUL
To: HUNTINGTON INGALLS INCORPORATED
Reel/Frame 050761/0070 →
Continuity (6)
Continuation In Part 16199780 · Nov 26, 2018
Continuation In Part 14729561 · Jun 3, 2015
Provisional Application 62036762 · Aug 13, 2014
Provisional Application 62025041 · Jul 16, 2014
Provisional Application 61997793 · Jun 9, 2014
Related Publication 20200050860A1 · Feb 13, 2020