IP Library Granted Patent US 11,061,467
Granted Patent B2
US 11,061,467 · App. 16/686,104 · Granted Jul 13, 2021

Electronic device displays an image of an obstructed target

Inventors: Philip Lyren (Bangkok, TH); Robert Lyren (Bangkok, TH)
G06F3/011H04W4/023
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,061,467
App. No.
16/686,104
Granted
Jul 13, 2021
Kind
B2
Abstract

An electronic device determines information about a target and provides the information to another electronic device that has an obstructed view of the target. The other electronic device displays an image of the target with an orientation and a location of the target.

Claims (51)

1. A method executed by one or more electronic devices in an electronic device system, the method comprising:

mapping, by the one or more electronic devices in the electronic device system, locations of objects at a geographical location;

determining, by the one or more electronic devices in the electronic device system and for one of the objects that is an obstructed object to a view of a camera of a smartphone at the geographical location, a location and an orientation of the obstructed object that the camera of the smartphone at the geographical location would capture if the obstructed object were not obstructed to the view of the camera;

displaying, on a display of the smartphone while the camera is pointed at the location of the obstructed object, a three-dimensional (3D) augmented reality (AR) image of the obstructed object at a location on the display of the smartphone where the obstructed object would exist if the obstructed object were not obstructed from the view of the camera; and

moving, based on movements captured of the obstructed object and while the camera is pointed at the location of the obstructed object and while the obstructed object remains obstructed from the view of the camera, the 3D AR image of the obstructed object to coincide with the movements of the obstructed object.

2. The method of claim 1 further comprising:

displaying, on the display of the smartphone while the camera is pointed at the location of the obstructed object, the 3D AR image of the obstructed object with an orientation that matches the orientation of the obstructed object that the display of the smartphone would display if the obstructed object were not obstructed from the view of the camera.

3. The method of claim 1 further comprising:

rotating, on the display of the smartphone while the camera is pointed at the location of the obstructed object, the 3D AR image of the obstructed object along a Z-axis so an orientation of the 3D AR image of the obstructed object along the Z-axis matches the orientation of the obstructed object along the Z-axis that the display of the smartphone would display if the obstructed object were not obstructed from the view of the camera.

4. The method of claim 1 further comprising:

simultaneously displaying, while the obstructed object remains obstructed from the view of the camera, the 3D AR image of the obstructed object on the display of the smartphone with a user and on a display of wearable electronic glasses (WEG) worn on a head of the user.

5. The method of claim 1 further comprising:

displaying, on the display of the smartphone while the camera is pointed at the location of the obstructed object and while the obstructed object remains obstructed from the view of the camera, the 3D AR image of the obstructed object along with a 3D AR image of a frame of reference that shows an area around the obstructed object.

6. The method of claim 1 further comprising:

capturing, with a second smartphone at the geographical location, information about the obstructed object that includes a global positioning system (GPS) location of the obstructed object, an image of the obstructed object, and a distance from the second smartphone to the obstructed object; and

sharing, with the smartphone, the information captured with the second smartphone.

7. The method of claim 1 further comprising:

displaying, on the display of the smartphone while the camera is pointed at the location of the obstructed object and while the obstructed object remains obstructed from the view of the camera, the 3D AR image of the obstructed object along with a distance from the smartphone to the obstructed object, a compass direction from the smartphone to the obstructed object, and an identity of the obstructed object.

8. A non-transitory computer readable storage medium storing instructions that cause one or more electronic devices to execute a method comprising:

building, based on information captured with one or more of smartphones and wearable electronic glasses (WEG) at a geographical location, augmented reality (AR) images of objects at the geographical location;

calculating, for one of the objects that is an obstructed object and not viewable through a camera of a smartphone while the camera is pointed in a direction to the obstructed object, a location where the obstructed object would appear on a display of the smartphone if the obstructed object were not obstructed;

displaying, on the display of the smartphone while the camera is pointed in the direction to the obstructed object, an AR image of the obstructed object at a location on the display of the smartphone where the obstructed object would appear on the display of the smartphone if the obstructed object were not obstructed; and

automatically enlarging, on the display of the smartphone while the obstructed object is obstructed to the smartphone, a size of the AR image of the obstructed object when a daze of a user is directed to the location on the display where the AR image of the obstructed object appears.

9. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

calculating, for the obstructed object, an orientation that the obstructed object would have on the display of the smartphone if the obstructed object were not obstructed; and

displaying, on the display of the smartphone while the camera is pointed in the direction to the obstructed object that is obstructed to the smartphone, the AR image of the obstructed with the orientation that the obstructed object would have on the display of the smartphone if the obstructed object were not obstructed.

10. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

displaying, on the display of the smartphone while the camera is pointed in the direction to the obstructed object that is obstructed to the smartphone, the AR image along with a distance from the smartphone to the obstructed object, a compass direction from the smartphone to the obstructed object, and an identity of the obstructed object.

11. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

tracking, with the smartphone, a gaze of a user holding the smartphone; and

automatically displaying, on the display of the smartphone while the obstructed object is obstructed to the smartphone, a direction to the obstructed object and a distance to the obstructed object when the gaze of the user is directed to the location on the display where the AR image of the obstructed object appears.

12. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

moving, on the display of the smartphone while the camera is pointed in the direction to the obstructed object and while the obstructed object is obstructed from being captured with the camera of the smartphone, the AR image of the obstructed object to coincide with real-time movements of the obstructed object.

13. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

rotating, on the display of the smartphone while the camera is pointed in the direction to the obstructed object and while the obstructed object is obstructed from being captured with the camera of the smartphone, the AR image of the obstructed object along a Z-axis so an orientation of the AR image of the obstructed object along the Z-axis matches an orientation of the obstructed object along the Z-axis that the display of the smartphone would display if the obstructed object were not obstructed.

14. A method comprising:

building, based on information captured with one or more portable electronic devices at a geographical location, augmented reality (AR) images of objects at the geographical location;

determining, for one of the objects that is an obstructed object and not viewable to a camera of a smartphone while the camera is pointed in a direction of the obstructed object, a location where the obstructed object would appear on a display of the smartphone if the obstructed object were not obstructed;

displaying, on the display of the smartphone while the camera is pointed in the direction of the obstructed object, an AR image of the obstructed object at a location on the display of the smartphone that coincides with where the obstructed object would appear on the display of the smartphone if the obstructed object were not obstructed;

receiving, with the smartphone, a command to track the obstructed object when a gaze of a user is directed to the AR image of the obstructed object; and

highlighting, on the display of the smartphone and in response to the voice command, the AR image of the obstructed object to show the obstructed object is being tracked.

15. The method of claim 14 further comprising:

calculating, by one or more processors, an orientation that the obstructed object would have on the display of the smartphone if the obstructed object were not obstructed; and

displaying, on the display of the smartphone while the camera is pointed in the direction of the obstructed object, the AR image of the obstructed with the orientation that the obstructed object would have on the display of the smartphone if the obstructed object were not obstructed.

16. The method of claim 14 further comprising:

tracking, with the smartphone, a gaze of a user holding the smartphone; and

automatically displaying, on the display of the smartphone, a direction to the obstructed object and a distance to the obstructed object when the gaze of the user is directed to the display where the AR image of the obstructed object appears.

17. The method of claim 14 further comprising:

determining, with the smartphone, a gaze of a user holding the smartphone;

sensing, with the smartphone, a gesture command to track the obstructed object when the gaze of the user is directed to the AR image of the obstructed object; and

highlighting, on the display of the smartphone and in response to the gesture command, the AR image of the obstructed object to show the obstructed object is being tracked.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2022
From: LYREN, PHILIP SCOTT; LYREN, ROBERT LOUIS
To: ADVANCED AR SYSTEMS, LLC
Reel/Frame 061589/0027 →
Continuity (5)
Continuation 16252729 · Jan 21, 2019
Continuation 15784230 · Oct 16, 2017
Continuation 14823542 · Aug 11, 2015
Substitution 62065672 · Oct 19, 2014
Related Publication 20200081522A1 · Mar 12, 2020