IP Library Granted Patent US 11,094,129
Granted Patent B2
US 11,094,129 · App. 16/672,413 · Granted Aug 17, 2021

Wearable electronic glasses display instructions as virtual hand gestures

Inventor: Philip Scott Lyren (Hong Kong, CN)
G06T19/006G02B27/0093G02B27/017G02B27/0172G06F1/163G06F3/012G06F40/51G02B2027/014G02B2027/0138G02B2027/0141G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,094,129
App. No.
16/672,413
Granted
Aug 17, 2021
Kind
B2
Abstract

First wearable electronic glasses (WEG) capture an image of a real obiect and provide the image to second WEG. The second WEG capture hand gestures performing a task on the obiect, and the first WEG displays the hand gestures that show how to perform the task on the real obiect being displayed through the first WEG.

Claims (45)

1. A method comprising: capturing, with a camera in first wearable electronic glasses (WEG) worn on a head of a first user, an image of a real object on which the first user will perform a task; transmitting, from the first WEG and to second WEG worn on a head of a second user, the image of the object; capturing, with a camera in the second WEG and while the second WEG displays the image of the real object, hand gestures of hands of the second user that move to perform the task on the image of the real object being displayed with the second WEG; displaying, with the first WEG and while the real object is visible through the display of the first WEG, virtual hands in augmented reality (AR) that move and replicate movements of the hand gestures of the second user that show the first user how to perform the task on the real object; and avoiding a collision between a perimeter of a field of view of the first WEG and the virtual hands being displayed by moving the virtual hands to an opposite side of the real object that is visible through the display of the first WEG.

2. The method of claim 1 further comprising:

displaying, with the first WEG and side-by-side with the real object, an AR object that emulates the real object with a size and a shape of the real object, wherein the virtual hands show performing the task on the AR object that is next to the real object.

3. The method of claim 1 further comprising:

detecting, with the first WEG, when the first user moves the real object but movements of the real object do not correctly perform the task; and

displaying, with the first WEG, a visual indication informing the first user that the movements of the real object did not correctly perform the task.

4. The method of claim 1 further comprising:

detecting, with the first WEG, when the first user moves the real object but movements of the real object do not correctly perform the task; and

providing, with the first WEG, a verbal indication informing the first user that the movements of the real object did not correctly perform the task.

5. The method of claim 1 further comprising:

displaying, with the first WEG and side-by-side with the real object, an AR object that emulates the real object;

moving the AR object to show how to perform a first sequence of the task on the real object; and

automatically moving the AR object to show how to perform a second sequence of the task in response to detecting with the first WEG when the first user moves the real object to emulate movements of the AR object performing the first sequence of the task.

6. The method of claim 1 further comprising:

displaying, with the first WEG and side-by-side with the real object, the AR object as a three-dimensional (3D) line drawing that emulates a size and a shape of the real object as seen through the first WEG.

7. The method of claim 1 further comprising:

receiving, at the first WEG and from the first user, a request to find the real object; and

displaying, with the first WEG and in response to the request, an AR image of a person that walks in a direction of the real object and guides the first person to the real object.

8. The method of claim 1 further comprising:

tracking, with eye tracking in the first WEG, a gaze of the first user to determine a location where the first user is reading in a textbook; and

automatically displaying, with the first WEG and in response to the first WEG detecting the gaze of the first user is looking at the location in the textbook, a three-dimensional (3D) AR image that coincides with subject matter at the location in the textbook.

9. A non-transitory computer readable storage medium storing instructions that cause one or more electronic devices to execute a method comprising:

capturing, with a camera in first wearable electronic glasses (WEG) worn on a head of a first user, an image of a real object on which the first user will perform a task;

displaying, with second WEG worn on a head of a second user, an augmented reality (AR) image that emulates the real object;

capturing, with a camera in the second WEG and while the second WEG displays the AR image that emulates the real object, hand gestures of hands of the second user that show how to perform the task on the AR image that emulates the real object;

displaying, with the first WEG and while the real object is visible through the display of the WEG, the AR image that emulates the real object and virtual three-dimensional (3D) hands that move and replicate movements of the hand gestures of the second user on the AR image that emulates the real object that show the first user how to perform the task on the real object; and

moving, by the first WEG, the AR image that emulates the real object from one side of the real object to an opposite side of the real object in response to determining movement of the first WEG will cause the AR image that emulates the real object to be outside a field of view of the first user if the AR image that emulates the real object remains at the first location.

10. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

displaying, with the first WEG, the AR image that emulates the real object side-by-side with the real object; and

detecting, with the first WEG, hand gestures of the first user to determine whether the hand gestures of the first user emulate movements of the virtual 3D hands performing the task on the AR image that emulates the real object.

11. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

decreasing, by the first WEG, a size of the AR image that emulates the real object being displayed at a location in response to determining movement of the first WEG will cause the AR image that emulates the real object to be outside the field of view of the first user if the AR image that emulates the real object remains at the location.

12. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

moving the AR image that emulates the real object from a first location in the field of view of the first WEG to a second location in the field of view of the first WEG in response to determining that movement of the first WEG will cause the first location to become too small to accommodate a size of the AR image that emulates the real object.

13. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:

automatically moving, by the first WEG, the AR image that emulates the real object from a first location to a second location in response to a perimeter of the field of view of the first WEG touching the AR image that emulates the real object.

14. An augmented reality (AR) system, comprising:

first wearable electronic glasses (WEG) that are worn on a head of a first user and include a display, a camera that captures an image of a real object seen through the display of the first WEG, and an interface that wirelessly transmits the image of the object; and

second WEG that are worn on a head of a second user and include a display that displays the image captured with the first WEG, a camera that captures hand gestures of the second user performing a task on the image captured with the first WEG, and an interface that wirelessly transmits the hand gestures to the first WEG,

wherein the real object that is visible through the display of the first WEG while the display of the first WEG displays the hand gestures as three-dimensional (3D) virtual hands that move and show the second user performing the task on the real object,

wherein the first WEG moves the 3D virtual hands from a first location to a second location in a field of view of the first WEG to counteract movement of the first WEG that will cause the 3D virtual hands to move outside the field of view if the 3D virtual hands remain at the first location.

15. The AR system of claim 14 , wherein the first WEG moves the 3D virtual hands to a different location within the field of view of the first WEG in order to avoid the 3D virtual hands from colliding with an edge of the field of view.

16. The AR system of claim 14 , wherein the first WEG reduces a size of the 3D virtual hands in order to prevent the 3D virtual hands from colliding with a perimeter of the field of view of the first WEG.

17. The AR system of claim 14 further comprising:

an eye tracker in the first WEG that tracks a direction of gaze of the first user and displays a virtual image of the real object in response to determining that the direction of gaze of the first user is directed to the real object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2022
From: LYREN, PHILIP SCOTT
To: ADVANCED AR SYSTEMS, LLC
Reel/Frame 061588/0975 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 9, 2021
From: LYREN, WILLIAM JAMES
To: LYREN, PHILIP SCOTT
Reel/Frame 056798/0214 →
Continuity (4)
Continuation 15423567 · Feb 2, 2017
Continuation 15017587 · Feb 16, 2016
Continuation 14044091 · Oct 2, 2013
Related Publication 20200066058A1 · Feb 27, 2020