IP Library Granted Patent US 10,019,843
Granted Patent B2
US 10,019,843 · App. 14/453,733 · Granted Jul 10, 2018

Controlling a near eye display

Inventors: Nadav Grossinger (Karmei Yosef, IL); Yair Alpem (Kfar Saba, IL)
Assignee: Facebook, Inc.
G06T19/006G06F1/163G06F3/011G06F3/017G06F3/0304
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,019,843
App. No.
14/453,733
Granted
Jul 10, 2018
Kind
B2
Abstract

A method and a system for controlling a near eye display using a virtual navigation space are provided herein. The system may include: a wearable near eye display; a sensor having a field of view, attached to the wearable near eye display and configured to capture a scene; a transmitter attached to the wearable near eye display said transmitter is configured to transmit a structured light pattern onto a navigation space, wherein the sensor is configured to capture reflections of the specified pattern coming from the navigation space; and a computer processor configured to analyze said reflections and control a visual indicator presented to a user over the wearable near eye display. The method implements the aforementioned logic without being limited to the architecture.

Claims (58)

1. An apparatus comprising:

a wearable near eye display;

a sensor attached to the wearable near eye display and configured to capture a scene;

a transmitter attached to the wearable near eye display, wherein the transmitter is configured to transmit a structured light pattern onto a navigation space and onto a hand of a user in the navigation space, and wherein the sensor is further configured to capture reflections of the structured light pattern coming from the navigation space; and

a computer processor configured to

analyze the reflections of the structured light pattern coming from the navigation space;

identify a gesture of the hand from a set of predefined gestures based on the reflections analysis, each gesture in the set of predefined gestures comprising a rotation or a posture change of the hand in the navigation space;

control a cursor presented to the user over the wearable near eye display according to the identified gesture of the hand, the cursor being a graphical pointing indication on the wearable near eye display interacting with content presented over the wearable near eye display;

invoke a corresponding action associated with the cursor over the wearable near eye display based on the identified gesture of the hand; and

move the cursor along an axis at the wearable near eye display based on the rotation of the hand along the axis at the navigation space, the hand remaining in a form of a first during the rotation.

2. The apparatus of claim 1 , wherein the computer processor is further configured to instruct the wearable near eye display to present a computer generated scene to the user providing a virtual reality environment to the user.

3. The apparatus of claim 1 , wherein the computer processor is further configured to instruct the wearable near eye display to present the captured scene to the user providing an augmented reality environment to the user.

4. The apparatus of claim 1 , wherein the computer processor is further configured to generate a 3D depth map based on the reflections and to analyze the 3D depth map.

5. The apparatus of claim 1 , wherein the computer processor is further configured to instruct the transmitter to illuminate only a portion of the navigation space comprising the hand.

6. The apparatus of claim 1 , wherein the computer processor is further configured to present, over the wearable near eye display, a computer generated scene with which the cursor interacts.

7. The apparatus of claim 1 , wherein the sensor and the transmitter are operable only within visible light bandwidth.

8. The apparatus of claim 1 , wherein the sensor that captures the scene and the sensor that captures the reflections is a single element.

9. The apparatus of claim 1 , wherein

the computer processor is further configured to move the cursor along one axis at the wearable near eye display based on the posture change of a finger of the hand along a defined axis at the navigation space.

10. The apparatus of claim 1 , wherein the computer processor is further configured to move the cursor along the axis at the wearable near eye display based on tilting a palm of the hand along a specified arc at the navigation space.

11. The apparatus of claim 1 , wherein a depth of field of the structured light pattern is selected to match a predefined location of the navigation space.

12. The apparatus of claim 1 , wherein the structured light pattern is continuous along a first axis and non-continuous along a second axis perpendicular to the first axis.

13. The apparatus of claim 1 , wherein the structured light pattern covers less than 10% of a field of view of the sensor.

14. A system comprising:

a transmitter attachable to a wearable near eye display;

a computer readable code executable on a computer processor and configured, when executed, to invoke the computer processor to:

instruct the transmitter to transmit a structured light pattern onto a navigation space and onto a hand of a user in the navigation space;

instruct a sensor attached to the wearable near eye display to capture a scene;

instruct the sensor to capture reflections of the structured light pattern coming from the navigation space;

analyze the reflections of the structured light pattern coming from the navigation space;

identify a gesture of the hand from a set of predefined gestures based on the reflections analysis, each gesture in the set of predefined gestures comprising a rotation or a posture change of the hand in the navigation space;

control a cursor presented to the user over the wearable near eye display according to the identified gesture of the hand, the cursor being a graphical pointing indication on the wearable near eye display interacting with content presented over the wearable near eye display;

invoke a corresponding action associated with the cursor over the wearable near eye display based on the identified gesture of the hand; and

move the cursor along an axis at the wearable near eye display based on the rotation of the hand along the axis at the navigation space, the hand remaining in a form of a first during the rotation.

15. A method comprising:

capturing a scene from a view point of a user;

transmitting a structured light pattern from the view point onto a navigation space and onto a hand of the user in the navigation space;

capturing reflections of the structured light pattern coming from the navigation space;

analyzing the reflections of the structured light pattern coming from the navigation space;

identifying a gesture of the hand from a set of predefined gestures based on the reflections analysis, each gesture in the set of predefined gestures comprising a rotation or a posture change of the hand in the navigation space;

controlling a cursor presented to the user over the wearable near eye display according to the identified gesture of the hand, the cursor being a graphical pointing indication on the wearable near eye display interacting with content presented over the wearable near eye display;

invoking a corresponding action associated with the cursor over the wearable near eye display based on the identified gesture of the hand; and

moving the cursor along an axis at the wearable near eye display based on the rotation of the hand along the axis at the navigation space, the hand remaining in a form of a first during the rotation.

16. The method of claim 15 , further comprising instructing the wearable near eye display to present a computer generated scene to the user providing a virtual reality environment to the user.

17. The method of claim 15 , further comprising instructing the wearable near eye display to present the captured scene to the user providing an augmented reality environment to the user.

18. The method of claim 15 , wherein analyzing the reflections comprises generating a 3D depth map based on the reflections and analyzing the 3D depth map.

19. The method of claim 15 , further comprising illuminating with the structured light pattern only a portion of the navigation space comprising the hand.

20. The method of claim 15 , further comprising presenting over the wearable near eye display, a computer generated scene with which the cursor interacts.

21. The method of claim 15 , further comprising moving the cursor along one axis at the wearable near eye display based on the posture change of a finger of the hand along a defined axis at the navigation space.

22. The method of claim 15 , further comprising moving the cursor along the axis at the wearable near eye display based on tilting a palm of the hand along a specified arc at the navigation space.

23. The apparatus of claim 1 , wherein the computer processor is further configured to identify the gesture of the hand from the set of predefined gestures by detecting segmentation of the structured light pattern reflected from the hand and captured by the sensor.

24. The apparatus of claim 1 , wherein the computer processor is further configured to:

generate a 3D depth map of the navigation space based on the reflections of the structured light pattern captured by the sensor; and

analyze segmentation in the 3D depth map of the navigation space to identify the gesture of the hand from the set of predefined gestures.

25. The method of claim 15 , further comprising identifying the gesture of the hand from the set of predefined gestures by detecting segmentation of the structured light pattern reflected from the hand.

26. The method of claim 15 , further comprising:

generating a 3D depth map of the navigation space based on the captured reflections of the structured light pattern; and

analyzing segmentation in the 3D depth map of the navigation space to identify the gesture of the hand from the set of predefined gestures.

Assignments (5)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2018
From: FACEBOOK, INC.
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047687/0942 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 4, 2016
From: OCULUS VR, LLC
To: FACEBOOK, INC.
Reel/Frame 040196/0790 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2016
From: PEBBLES LTD.
To: OCULUS VR, LLC
Reel/Frame 037545/0701 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2015
From: GROSSINGER, NADAV; ALPERN, YAIR
To: PEBBLES LTD.
Reel/Frame 035946/0458 →
Continuity (2)
Provisional Application 61863510 · Aug 8, 2013
Related Publication 20150042680A1 · Feb 12, 2015
Cited By (2)
US 12,572,213 US 12,619,316