IP Library Granted Patent US 9,507,411
Granted Patent B2
US 9,507,411 · App. 13/497,589 · Granted Nov 29, 2016

Hand tracker for device with display

Inventors: Nadav Grossinger (Tel-Aviv, IL); Israel Grossinger (Karmei Yoseef, IL); Nitay Romano (Geva Binyamin, IL)
Assignee: Facebook, Inc.
G06F3/005G06F3/017
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,507,411
App. No.
13/497,589
Granted
Nov 29, 2016
Kind
B2
Abstract

An apparatus for remote hand control of a computer device, the apparatus comprising: a hand tracker, configured to track movement of a hand of a user of the computer device, an image presenter, associated with the hand tracker, configured to present a first image representative of the hand and a second image to the user, and to move the first image in concert with the tracked movement, and a computer controller, associated with the image presenter and configured to control the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into the interaction.

Claims (88)

1. An apparatus for remote hand control of a computer device, the apparatus comprising:

a hand tracker, configured to track movement of a hand of a user of the computer device;

an image presenter, associated with said hand tracker, configured to present a first image representative of the hand and a second image to the user, and to move the first image in concert with the tracked movement; and

a computer controller, associated with said image presenter and configured to control the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction, wherein

said hand tracker is further configured to track the movement by detecting a segmentation of a continuous feature in a light pattern projected onto the hand, the light pattern as projected onto a flat surface having said continuous feature in a first direction and being non-continuous in a direction substantially perpendicular to the first direction; and

said hand tracker detects segmentation of said continuous feature based on changes in light intensity profiles of reflected light along said continuous feature.

2. The apparatus of claim 1 , wherein said hand tracker is further configured to use bi-dimensional video data, for tracking the movement of the hand.

3. The apparatus of claim 1 , wherein the light pattern comprises a plurality of stripes arranged substantially in parallel to each other, and said hand tracker is further configured to identify in the light pattern, at least one cluster of stripe segments created by segmentation of stripes by a respective digit of the hand, and to track movement of the digit by tracking only the cluster of stripe segments created by segmentation of stripes by the digit.

4. The apparatus of claim 3 , wherein said hand tracker is further configured to identify a depth movement of the digit, by detecting a change in the number of stripe segments in the tracked cluster.

5. The apparatus of claim 1 , further comprising a light projector, configured to project a light pattern having a continuous feature in a first direction and a non-continuous feature in a direction substantially perpendicular to the first direction, onto the hand, and a camera, associated with said hand tracker and configured to capture the first image, by capturing an image of the hand projected with the light pattern.

6. The apparatus of claim 5 , wherein said light projector is remote from the computer device.

7. The apparatus of claim 5 , wherein said camera is remote from the computer device.

8. The apparatus of claim 5 , wherein said light projector comprises a source of light configured to emit a light beam, and a micro structured element configured to modify the light beam, for generating the light pattern.

9. The apparatus of claim 8 , wherein said micro structured element is further configured to convert the light beam into a light beam having a cross sectional intensity profile changing along the light beam.

10. The apparatus of claim 1 , further comprising a gesture recognizer, associated with said hand tracker, configured to detect a predefined gesture in the tracked movement of the hand, wherein said computer controller is further configured to control the computer device in a manner predefined for the detected gesture.

11. The apparatus of claim 10 , wherein said image presenter is further configured to carry out one member of the group consisting of: aligning of the first image into a predefined position, upon the detection of the predefined gesture, resizing the first image, upon the detection of the predefined gesture, and resizing the second image, upon the detection of the predefined gesture.

12. The apparatus of claim 1 , wherein said computer controller is a part of one member of the group consisting of a mobile phone, a device installed in a car and a musical instrument.

13. The apparatus of claim 1 , wherein said hand tracker is remote from the computer device.

14. The apparatus of claim 1 , wherein the second image is representative of a computer input device.

15. The apparatus of claim 1 , wherein the continuous feature in the projected light pattern has a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

16. The apparatus of claim 1 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

17. An apparatus for remote hand control of a computer device, the apparatus comprising:

a hand tracker, configured to track movement of a hand of a user of the computer device;

an image presenter, associated with said hand tracker, configured to present a first image representative of the hand and a second image to the user, and to move the first image in concert with the tracked movement; and

a computer controller, associated with said image presenter and configured to control the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction, wherein

said hand tracker is configured to project a light pattern, said light pattern comprising a plurality of stripes arranged substantially in parallel to each other;

said hand tracker being further configured to identify in the light pattern, at least one cluster of stripe segments created by segmentation of stripes by a palm of the hand, and to track movement of the hand by tracking the cluster of stripe segments created by segmentation of stripes by the palm; and

said hand tracker detects said stripe segments based on changes in light intensity profiles of reflected light along said stripes.

18. The apparatus of claim 17 , wherein the plurality of stripes in said projected light pattern have a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

19. The apparatus of claim 17 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

20. An apparatus for remote hand control of a computer device, the apparatus comprising:

a hand tracker, configured to track movement of a hand of a user of the computer device;

an image presenter, associated with said hand tracker, configured to present a first image representative of the hand and a second image to the user, and to move the first image in concert with the tracked movement; and

a computer controller, associated with said image presenter and configured to control the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction; wherein

said hand tracker is further configured to carry out at least one member of a group consisting of identifying and tracking an object, said tracking being carried out by detecting a shift in position of a notation along a continuous feature in a light pattern projected onto the hand; and

said hand tracker detects a shift in position of said notation along said continuous feature based on changes in light intensity profiles of reflected light along said continuous feature.

21. The apparatus of claim 20 , wherein the continuous feature in the projected light pattern has a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

22. The apparatus of claim 20 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

23. A computer implemented method for remote hand control of a computer device, the method comprising steps the computer device is programmed to perform, the steps comprising:

a) tracking movement of a hand of a user of the computer device;

b) presenting a first image representative of the hand and a second image to the user, and moving the first image in concert with the tracked movement; and

c) controlling the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction; wherein

said tracking of the movement of the hand further comprises detecting a segmentation of a continuous feature in a light pattern projected onto the hand, the light pattern as projected onto a flat surface having said continuous feature in a first direction and being non-continuous in a direction substantially perpendicular to the first direction; and

said detecting a segmentation of said continuous feature is based on changes in light intensity profiles of reflected light along said continuous feature.

24. The method of claim 23 , wherein said tracking of the movement of the hand is carried out using bi-dimensional video data.

25. The method of claim 23 , wherein said tracking of the movement of the hand further comprises detecting a shift in position of a notation along a continuous feature in a light pattern projected onto the hand.

26. The computer implemented method of claim 23 , wherein the continuous feature in the projected light pattern has a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

27. The computer implemented method of claim 23 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

28. A computer implemented method for remote hand control of a computer device, the method comprising steps the computer device is programmed to perform, the steps comprising:

a) tracking movement of a hand of a user of the computer device;

b) presenting a first image representative of the hand and a second image to the user, and moving the first image in concert with the tracked movement; and

c) controlling the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction;

wherein

said tracking further comprises projecting a light pattern, and

the light pattern comprises a plurality of stripes arranged substantially in parallel to each other,

the method further comprising

identifying in the light pattern, at least one cluster of stripe segments created by segmentation of stripes by a respective digit of the hand, said segmentation of stripes based on changes in light intensity profiles of reflected light along said stripes; and

tracking movement of the digit by tracking the cluster of stripe segments created by segmentation of stripes by the digit.

29. The method of claim 28 , further comprising identifying a depth movement of the digit, by detecting a change in the number of stripe segments in the tracked cluster.

30. The computer implemented method of claim 28 , wherein the plurality of stripes arranged substantially in parallel to each other in the projected light pattern each have a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

31. The computer implemented method of claim 28 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

32. A computer implemented method for remote hand control of a computer device, the method comprising steps the computer device is programmed to perform, the steps comprising:

a) tracking movement of a hand of a user of the computer device;

b) presenting a first image representative of the hand and a second image to the user, and moving the first image in concert with the tracked movement; and

c) controlling the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction; wherein

said tracking comprises projecting a light pattern, and

the light pattern comprises a plurality of stripes arranged substantially in parallel to each other,

the method further comprising

identifying in the light pattern, at least one cluster of stripe segments created by segmentation of stripes by a palm of the hand, said segmentation of stripes based on changes in light intensity profiles of reflected light along said stripes; and

tracking movement of the hand by tracking the cluster of stripe segments created by segmentation of stripes by the palm.

33. The computer implemented method of claim 32 , wherein the plurality of stripes arranged substantially in parallel to each other in the projected light pattern each have a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

34. The computer implemented method of claim 32 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

35. A non-transitory computer readable medium storing computer executable instructions for performing steps of remote hand control of a computer device, the steps comprising:

a) tracking movement of a hand of a user of the computer device;

b) presenting a first image representative of the hand and a second image to the user, and moving the first image in concert with the tracked movement; and

c) controlling the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction, wherein

said tracking comprises detecting a segmentation of a continuous feature in a light pattern projected onto the hand, the light pattern as projected onto a flat surface having said continuous feature in a first direction and being non-continuous in a direction substantially perpendicular to the first direction; and

said tracking detects segmentation of said continuous feature based on changes in light intensity profiles of reflected light along said continuous feature.

36. The non-transitory computer readable medium of claim 35 , the continuous feature in the projected light pattern has a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

37. The non-transitory computer readable medium of claim 35 , wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

38. An apparatus for remote hand control of a computer device, the apparatus comprising:

a hand tracker, configured to track a hand of a user of the computer device;

an image presenter, associated with said hand tracker, configured to present a first image representative of the hand and a second image to the user, and to move the first image in concert with the tracking; and

a computer controller, associated with said image presenter and configured to control the computer device in accordance with an interaction between the first image and the second image, thereby allowing the user to control the computer device, by moving the hand for bringing the first image and the second image into said interaction, wherein

said hand tracker is further configured to track the hand by detecting a segmentation of a continuous feature in a light pattern projected onto the hand, the light pattern as projected onto a flat surface having said continuous feature in a first direction and being non-continuous in a direction substantially perpendicular to the first direction; and

said hand tracker detects segmentation of said continuous feature based on changes in light intensity profiles of reflected light along said continuous feature.

39. The apparatus of claim 38 , wherein the continuous feature in the projected light pattern has a changing light intensity profile in a direction perpendicular to a propagation direction of the light pattern.

40. The apparatus of claim 38 wherein the projected light pattern has a cross-sectional intensity profile which changes in a direction that the light propagates towards an object.

Assignments (6)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2018
From: FACEBOOK, INC.
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047687/0942 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 4, 2016
From: OCULUS VR, LLC
To: FACEBOOK, INC.
Reel/Frame 040196/0790 →
CHANGE OF NAME Recorded Feb 1, 2016
From: PEBBLESTECH LTD.
To: PEBBLES LTD.
Reel/Frame 037678/0666 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2016
From: PEBBLES LTD.
To: OCULUS VR, LLC
Reel/Frame 037545/0701 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2012
From: GROSSINGER, NADAV; GROSSINGER, ISRAEL; ROMANO, NITAY
To: PEBBLESTECH LTD.
Reel/Frame 027968/0463 →
Continuity (2)
Provisional Application 61244473 · Sep 22, 2009
Related Publication 20120194561A1 · Aug 2, 2012