IP Library Granted Patent US 10,055,890
Granted Patent B2
US 10,055,890 · App. 15/266,136 · Granted Aug 21, 2018

Augmented reality for wireless mobile devices

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,055,890
App. No.
15/266,136
Granted
Aug 21, 2018
Kind
B2
Abstract

A model includes model layers on which a wireframe representation of objects located at geographic coordinates is stored in a memory such that surface detail of the objects increases from a base model layer to an uppermost model layer. Digital data layers stored in the memory that encompass digital coordinates corresponding with the geographic coordinates. Digital content for augmenting scenes is stored on pre-selected digital data layers at pre-selected digital coordinates on those layers. One or more of the digital data layers are logically linked with one or more of the model layers. When the location and spatial orientation of a mobile device in which a scene is viewed is received, the digital content on the digital data layer logically linked to one of the model layers is transmitted over a wireless communication channel to the mobile device.

Claims (70)

1. A method for reality augmentation comprising:

storing, in a memory device:

a model representing real world objects located at corresponding geographic coordinates of a geographic coordinate system, the model comprising a data structure of model layers in which a wireframe representation of the real world objects is represented in a base model layer and in which surface detail of the real world objects increases from the base model layer to an uppermost model layer;

a data structure having a plurality of digital data layers encompassing digital coordinates of a digital coordinate system corresponding with respective geographic coordinates of the geographic coordinate system; and

digital content on pre-selected one or more of the digital data layers at pre-selected digital coordinates thereon, the digital content comprising data for augmenting scenes containing the real world objects at the respective locations;

logically linking one or more of the digital data layers with one or more of the model layers to locate the digital content within the scenes based upon the digital coordinates and the model;

receiving the location and spatial orientation of a mobile device in which a scene is viewed by the mobile device that overlaps with the geographic coordinates; and

transmitting, over a wireless communication channel to the mobile device, the digital content on the digital data layer logically linked to a selected one of the model layers for geographic coordinates that include the scene viewed by the mobile device to augment the scene viewed by the mobile device using the digital content.

2. The method of claim 1 , wherein transmitting the digital content further comprises:

transmitting the digital content on the base model layer as the selected model layer and the wireframe representation of the real world objects in the scene viewed by the mobile device.

3. The method of claim 2 , wherein transmitting the digital content further comprises;

transmitting the digital content and the wireframe representation of the real world objects in a predetermined area surrounding the location of the mobile device.

4. The method of claim 1 , further comprising:

identifying key features in image data of the real world objects, each of the key features being located at known geographic coordinates of the geographic coordinate system; and

constructing the wireframe representation of the image data from data points at the digital coordinates of the digital coordinate system corresponding to the known geographic coordinates of identified key features; and

storing the wireframe model on the base model layer.

5. The method of claim 1 , further comprising:

retrieving the base model layer corresponding with image data presented in the view of the mobile device in response to receiving the image data and the location of the mobile device;

registering the wireframe representation of the retrieved base model layer on the received image data; and

transmitting the digital content at the digital coordinates on the digital data layer corresponding with the geographic coordinates of the registered image data.

6. The method of claim 1 , further comprising:

determining bandwidth of the wireless communication channel; and

transmitting, over the wireless communication channel, one of the model layers having greater surface detail than the wireframe representation on the base model layer with the digital content on the digital data layer logically linked therewith based on the determined bandwidth.

7. A server apparatus for augmenting reality comprising:

a memory device configured to store:

a model representing real world objects located at corresponding geographic coordinates of a geographic coordinate system, the model comprising a data structure of model layers in which a wireframe representation of the real world objects is represented in a base model layer and in which surface detail of the real world objects increases from the base model layer to an uppermost model layer;

a data structure having a plurality of digital data layers encompassing digital coordinates of a digital coordinate system corresponding with respective geographic coordinates of the geographic coordinate system; and

digital content on pre-selected one or more of the digital data layers at pre-selected digital coordinates thereon, the digital content comprising data for augmenting scenes containing the real world objects at the respective locations; and

a processor configured to:

logically link one or more of the digital data layers with one or more of the model layers to locate the digital content within the scenes based upon the digital coordinates and the model;

receive the location and spatial orientation of a mobile device in which a scene is viewed by the mobile device that overlaps with the geographic coordinates; and

transmit, over a wireless communication channel to the mobile device, the digital content on the digital data layer logically linked to a selected one of the model layers for geographic coordinates that include the scene viewed by the mobile device to augment the scene viewed by the mobile device using the digital content.

8. The server apparatus of claim 7 , wherein the processor is further configured to:

transmit the digital content on the base model layer as the selected model layer and the wireframe representation of the real world objects in the scene viewed by the mobile device.

9. The server apparatus of claim 8 , wherein the processor is further configured to:

transmit the digital content and the wireframe representation of the real world objects in a predetermined area surrounding the location of the mobile device.

10. The server apparatus of claim 7 , wherein the processor is further configured to:

identify key features in image data of the real world objects, each of the key features being located at known geographic coordinates of the geographic coordinate system; and

construct the wireframe representation of the image data from data points at the digital coordinates of the digital coordinate system corresponding to the known geographic coordinates of identified key features; and

store the wireframe model on the base model layer.

11. The server apparatus of claim 7 , wherein the processor is further configured to:

retrieve the base model layer corresponding with image data presented in the view of the mobile device in response to receiving the image data and the location of the mobile device;

register the wireframe representation of the retrieved base model layer on the received image data; and

transmit the digital content at the digital coordinates on the digital data layer corresponding with the geographic coordinates of the registered image data.

12. The server apparatus of claim 7 , wherein the processor is further configured to:

determine bandwidth of the wireless communication channel; and

transmit, over the wireless communication channel, one of the model layers having greater surface detail than the wireframe representation on the base model layer with the digital content on the digital data layer logically linked therewith based on the determined bandwidth.

13. A non-transitory, tangible computer-readable medium having stored thereon processor instructions that, when executed by a processor, causes the processor to:

store, in a memory device:

a model representing real world objects located at corresponding geographic coordinates of a geographic coordinate system, the model comprising a data structure of model layers in which a wireframe representation of the real world objects is represented in a base model layer and in which surface detail of the real world objects increases from the base model layer to an uppermost model layer;

a data structure having a plurality of digital data layers encompassing digital coordinates of a digital coordinate system corresponding with respective geographic coordinates of the geographic coordinate system; and

digital content on pre-selected one or more of the digital data layers at pre-selected digital coordinates thereon, the digital content comprising data for augmenting scenes containing the real world objects at the respective locations;

logically link one or more of the digital data layers with one or more of the model layers to locate the digital content within the scenes based upon the digital coordinates and the model;

receive the location and spatial orientation of a mobile device in which a scene is viewed by the mobile device that overlaps with the geographic coordinates; and

transmit, over a wireless communication channel to the mobile device, the digital content on the digital data layer logically linked to a selected one of the model layers for geographic coordinates that include the scene viewed by the mobile device to augment the scene viewed by the mobile device using the digital content.

14. The computer-readable medium of claim 13 , wherein the processor instructions include processor instructions that cause the processor to:

transmit the digital content on the base model layer as the selected model layer and the wireframe representation of the real world objects in the scene viewed by the mobile device.

15. The computer-readable medium of claim 14 , wherein the processor instructions include processor instructions that cause the processor to:

transmit the digital content and the wireframe representation of the real world objects in a predetermined area surrounding the location of the mobile device.

16. The computer-readable medium of claim 13 , wherein the processor instructions include processor instructions that cause the processor to:

identify key features in image data of the real world objects, each of the key features being located at known geographic coordinates of the geographic coordinate system; and

construct the wireframe representation of the image data from data points at the digital coordinates of the digital coordinate system corresponding to the known geographic coordinates of identified key features; and

store the wireframe model on the base model layer.

17. The computer-readable medium of claim 13 , wherein the processor instructions include processor instructions that cause the processor to:

retrieve the base model layer corresponding with image data presented in the view of the mobile device in response to receiving the image data and the location of the mobile device;

register the wireframe representation of the retrieved base model layer on the received image data; and

transmit the digital content at the digital coordinates on the digital data layer corresponding with the geographic coordinates of the registered image data.

18. The computer-readable medium of claim 13 , wherein the processor instructions include processor instructions that cause the processor to:

determine bandwidth of the wireless communication channel; and

transmit, over the wireless communication channel, one of the model layers having greater surface detail than the wireframe representation on the base model layer with the digital content on the digital data layer logically linked therewith based on the determined bandwidth.

Assignments (5)
SECURITY INTEREST Recorded Feb 21, 2024
From: ELBIT SYSTEMS OF AMERICA, LLC; SPARTON CORPORATION; SPARTON DELEON SPRINGS, LLC; LOGOS TECHNOLOGIES LLC; ELBITAMERICA, INC.; KMC SYSTEMS, INC.
To: CAPITAL ONE, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 066642/0935 →
RELEASE OF SECURITY INTEREST Recorded Feb 21, 2024
From: WELLS FARGO BANK, NATIONAL ASSOCIATION
To: ELBIT SYSTEMS OF AMERICA, LLC
Reel/Frame 066644/0612 →
CHANGE OF NAME Recorded Sep 17, 2019
From: HARRIS CORPORATION
To: L3HARRIS TECHNOLOGIES, INC.
Reel/Frame 050409/0288 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2019
From: L3HARRIS TECHNOLOGIES, INC.; EAGLE TECHNOLOGY, LLC
To: ELBIT SYSTEMS OF AMERICA, LLC
Reel/Frame 050375/0008 →
SECURITY INTEREST Recorded Sep 13, 2019
From: ELBIT SYSTEMS OF AMERICA, LLC
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 050375/0425 →