IP Library Granted Patent US 11,657,577
Granted Patent B2
US 11,657,577 · App. 16/679,263 · Granted May 23, 2023

Wearable electronic glasses display an augmented reality (AR) image to show a task

Inventor: Philip Scott Lyren (Hong Kong, CN)
G06T19/006G02B27/0093G02B27/017G02B27/0172G06F1/163G06F3/012G06F40/51G02B2027/014G02B2027/0138G02B2027/0141G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,657,577
App. No.
16/679,263
Granted
May 23, 2023
Kind
B2
Abstract

A wearable electronic device (WED) worn on a head of a first user displays a virtual location with virtual objects. The WED receives a task on one of the AR objects that corresponds to a real object at a real location. Wearable electronic glasses (WEG) worn on a head of a second user display an augmented reality (AR) image that shows the task to be performed on the real object.

Claims (53)

1. A method comprising:

displaying, on a display of a portable electronic device (PED) in a hand of a first user, a virtual location with virtual objects that emulates a real location with real objects that is remotely located from the PED;

tracking, with the PED in the hand of the first user, hand movements of the first user interacting with one of the virtual objects being displayed on the display of the PED such that the hand movements interacting with the one of the virtual objects show a task to be performed on one of the real objects that corresponds to the one of the virtual objects;

tracking, with wearable electronic glasses (WEG) worn on a head of a second user located at the real location, a direction of gaze of the second user; and

displaying, with the WEG worn on the head of the second user and in response to the direction of gaze of the second user being at the one of the real objects, an augmented reality (AR) image of the hand movements and the virtual objects that shows the task to be performed that was provided by the first user on the one of the virtual objects.

2. The method of claim 1 further comprising:

receiving, at a user interface (UI) of the PED and from the first user, an action that highlights the one of the virtual objects; and

displaying, with the WEG worn on the head of the second user and in response to the action from the first user, the AR image that highlights the one of the real objects when the direction of gaze of the second user is at the one of the real objects.

3. The method of claim 1 further comprising:

receiving, at the one of the virtual objects being displayed on the display of the PED and from the first user, an action that moves the one of the virtual objects from a first location at the virtual location to a second location at the virtual location; and

displaying, with the WEG worn on the head of the second user and in response to the action from the first user, the AR image that moves from a first location at the real location to a second location at the real location,

wherein the first location at the real location corresponds to the first location at the virtual location, and the second location at the real location correspond to the second location at the virtual location.

4. The method of claim 1 further comprising:

receiving, at the one of the virtual objects being displayed on the display of the PED and from the first user, the task that includes a sequence of actions showing where to move the one of the real objects at the real location; and

displaying, with the WEG worn on the head of the second user, the AR image that moves to different locations at the real location and shows the sequence of actions showing where to move the one of the real objects.

5. The method of claim 1 further comprising:

displaying, on the display of the PED in the hand of the first user, video of the real location received from a camera in the WEG;

receiving, at a user interface (UI) of the PED and from the first user, an action that selects a real object shown in the video of the real location; and

displaying, with the WEG, the AR image over the real object to visually identify the real object selected by the first user.

6. The method of claim 1 further comprising:

receiving, at a user interface (UI) of the PED, an action from the first user that moves the one of the virtual objects; and

moving, with the WEG, the AR image in real-time with movements of the one of the virtual objects to show the task to be performed.

7. The method of claim 1 further comprising:

moving a location where the AR image is displayed with the WEG in order to avoid a collision between the AR image and a field of view of the WEG.

8. The method of claim 1 further comprising:

moving a location of the AR image from one side of the one of the real objects to an opposite side of the one of the real objects in response to movement of the WEG that causes an available space on the one side of the real objects to be too small for the AR image.

9. A non-transitory computer readable storage medium storing instructions that cause one or more electronic devices to execute a method comprising:

displaying, with a wearable electronic device (WED) worn on a head of a first user, a virtual location having virtual objects that emulates a real location having real objects;

receiving, at one of the virtual objects being displayed with the WED and from the first user, hand gestures of the first user interacting with the one of the virtual objects such that the hand gestures interacting with the one of the virtual objects show a task to be performed on one of the real objects that corresponds to the one of the virtual objects; and

displaying, with wearable electronic glasses (WEG) worn on a head of a second user, an augmented reality (AR) image of the hand gestures that moves at the real location and shows the task to be performed on the one of the real objects that was provided by the first user on the one of the virtual objects.

10. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

tracking, with the WEG worn on the head of the second user located at the real location, a direction of gaze of the second user; and

displaying, with the WEG worn on the head of the second user located at the real location, the AR image over the one of the real objects in response to the direction of gaze of the second user being at the one of the real objects.

11. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

moving, with the WEG worn on the head of the second user, the AR image to correspond with real-time movements of the one of the virtual objects being displayed on the WED and being moved by the first user.

12. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

highlighting, with the WEG worn on the head of the second user, the one of real objects with the AR image in response to a user interface (UI) of the WED receiving highlighting of the one of the virtual objects being displayed with the WED.

13. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

receiving, at the one of the virtual objects being displayed on the WED and from the first user, the task that includes a sequence of actions showing where to move the one of the real objects at the real location; and

displaying, with the WEG worn on the head of the second user, the AR image that moves to different locations at the real location and shows the sequence of actions showing where to move the one of the real objects.

14. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

displaying, with the WED, the virtual location having the virtual objects in virtual reality (VR);

capturing, with a camera in the WEG worn on the head of the second user, video of the real location; and

displaying, with the WED worn on the head of the first user, the video of the real location.

15. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

moving a location where the AR image is displayed with the WEG in order to avoid a collision between the AR image and a field of view of the WEG.

16. An electronic system, comprising:

a wearable electronic device (WED) worn on a head of a first user and including a display that displays a virtual reality (VR) location having VR objects that emulates a real location having real objects and including a user interface (UI) that receives from the first user hand gestures interacting with the VR objects that show how to perform a task on one of the real objects that corresponds to the one of the VR objects; and

wearable electronic glasses (WEG) worn on a head of a second user and including a display that displays an augmented reality (AR) image of the hand gestures that moves at the real location and shows the task to be performed on the one of the real objects that was provided by the first user on the one of the VR objects.

17. The electronic system of claim 16 , wherein the WEG further includes eye tracking that tracks a direction of gaze of the second user, and the WEG displays that the AR image over the one of the real objects in response to the direction of gaze of the second user being at the one of the real objects.

18. The electronic system of claim 16 , wherein the WEG further includes eye tracking that tracks a direction of gaze of the second user, and the WEG automatically moves the AR image through a sequence of actions showing the task to be performed on the one of the real objects in response to the direction of gaze of the second user being at the one of the real objects.

19. The electronic system of claim 16 , wherein the WEG further includes eye tracking that tracks a direction of gaze of the second user, and the WEG automatically plays a voice message from the first user in response to the direction of gaze of the second user being at the one of the real objects.

20. The electronic system of claim 16 , wherein the WEG automatically moves a where the AR image is being displayed in order to avoid a collision between the AR image and a perimeter of a field of view of the WEG.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2022
From: LYREN, PHILIP SCOTT
To: ADVANCED AR SYSTEMS, LLC
Reel/Frame 061588/0975 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 9, 2021
From: LYREN, WILLIAM JAMES
To: LYREN, PHILIP SCOTT
Reel/Frame 056798/0214 →
Continuity (4)
Continuation 15423567 · Feb 2, 2017
Continuation 15017587 · Feb 6, 2016
Continuation 14044091 · Oct 2, 2013
Related Publication 20200074746A1 · Mar 5, 2020