IP Library Granted Patent US 10,429,940
Granted Patent B2
US 10,429,940 · App. 15/856,603 · Granted Oct 1, 2019

Device and method for recognizing hand gestures using time-of-flight sensing

Inventor: Olivier Pothier (Sceaux, FR)
Assignee: STMicroelectronics SA
G06F3/017G01S17/58G06F3/0325
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,429,940
App. No.
15/856,603
Granted
Oct 1, 2019
Kind
B2
Abstract

An electronic device includes at least one laser source configured to direct laser radiation toward a user's hand. Laser detectors are configured to receive reflected laser radiation from the user's hand. A controller is coupled to the at least one laser source and laser detectors and configured to determine a set of distance values to the user's hand for each respective laser detector and based upon a time-of-flight of the laser radiation. The controller also determines a hand gesture from among a plurality of possible hand gestures based upon the sets of distance values using Bayesian probabilities.

Claims (46)

1. An electronic device, comprising:

a plurality of laser sources configured to direct laser radiation toward an object external to the electronic device;

a plurality of laser detectors configured to receive reflected laser radiation from the object; and

a controller coupled to the plurality of laser sources and the plurality of laser detectors, wherein the controller is configured to:

determine, for each respective laser detector of the plurality of laser detectors, a respective elapsed time between transmission of the laser radiation from the plurality of laser sources and reception of the reflected laser radiation at the respective laser detector;

determine, for each respective laser detector, a respective distance between the respective laser detector and the object;

determine, for each respective laser detector, a respective confusion matrix based on the respective distance, the respective confusion matrix being indicative of probabilities of a plurality of possible hand gestures as determined from a position of the respective laser detector, wherein respective confusion matrices of the plurality of laser detectors collectively form a set of confusion matrices; and

determine a hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices.

2. The electronic device according to claim 1 , wherein the controller is configured to determine the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices by weighting each respective distance by its corresponding confusion matrix.

3. The electronic device according to claim 1 , wherein the controller is configured to determine the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices by:

determining, for each respective laser detector, a respective probable hand gesture based on the respective confusion matrix of the respective laser detector, wherein the respective probable hand gestures collectively form a set of probable hand gestures; and

selecting the hand gesture from the set of probable hand gestures based on a majority vote.

4. The electronic device according to claim 1 , further comprising a memory coupled to the controller and configured to store the sets of confusion matrices.

5. The electronic device according to claim 1 , wherein the controller is configured to determine the hand gesture as at least one of a single tap, a double tap, a page flip, a single wipe, a double wipe, and a rotation.

6. The electronic device according to claim 1 , wherein each respective laser detector comprises a single photon avalanche diode (SPAD) detector.

7. The electronic device according to claim 6 , wherein the SPAD detector comprises an array of single photon avalanche diodes.

8. The electronic device according to claim 6 , wherein the plurality of laser sources, the SPAD detector, and the controller are formed as a single integrated circuit (IC).

9. The electronic device according to claim 1 , wherein the plurality of laser sources comprises a vertical-cavity surface-emitting laser (VCSEL).

10. The electronic device according to claim 1 , wherein the plurality of laser sources comprises an infrared (IR) laser source.

11. The electronic device according to claim 1 , wherein the plurality of laser sources has a pitch of about 40 micrometers.

12. An electronic device, comprising:

a laser source configured to direct laser radiation toward a user's hand, the laser source comprising a vertical-cavity surface-emitting laser (VCSEL);

a plurality of single photon avalanche diode (SPAD) detectors configured to receive reflected laser radiation from the user's hand; and

a controller coupled to the laser source and plurality of SPAD detectors, wherein the controller is configured to:

determine, for each respective SPAD detector of the plurality of SPAD detectors, a respective elapsed time between transmission of the laser radiation from the laser source and reception of the reflected laser radiation at the respective SPAD detector;

determine, for each respective SPAD detector, a respective distance between the respective SPAD detector and the user's hand;

determine, for each respective SPAD detector, a respective confusion matrix based on the respective distance, the respective confusion matrix being indicative of probabilities of a plurality of possible hand gestures as determined from a position of the respective SPAD detector, wherein respective confusion matrices of the plurality of SPAD detectors collectively form a set of confusion matrices; and

determine a hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices.

13. The electronic device according to claim 12 , wherein the controller is configured to determine the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices by weighting each respective distance by its corresponding confusion matrix.

14. The electronic device according to claim 12 , wherein the controller is configured to determine the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices by:

determining, for each respective SPAD detector, a respective probable hand gesture based on the respective confusion matrix of the respective SPAD detector, wherein the respective probable hand gestures collectively form a set of probable hand gestures; and

selecting the hand gesture from the set of probable hand gestures based on a majority vote.

15. The electronic device according to claim 12 , further comprising a memory coupled to the controller and configured to store the sets of confusion matrices.

16. The electronic device according to claim 12 , wherein the controller is configured to determine the hand gesture as at least one of a single tap, a double tap, a page flip, a single wipe, a double wipe, and a rotation.

17. The electronic device according to claim 12 , wherein the laser source, the plurality of SPAD detectors, and the controller are formed as a single integrated circuit (IC).

18. A method of determining a hand gesture, the method comprising:

directing laser radiation from a laser source toward a user's hand;

receiving, with a plurality of laser detectors, reflected laser radiation from the user's hand;

determining, using a controller coupled to the laser source and the plurality of laser detectors and for each respective laser detector of the plurality of laser detectors, a respective elapsed time between transmission of the laser radiation from the laser source and reception of the reflected laser radiation at the respective laser detector;

determining, using the controller and for each respective laser detector, a respective distance between the respective laser detector and the user's hand;

determining, using the controller and for each respective laser detector, a respective confusion matrix based on the respective distance, the respective confusion matrix being indicative of probabilities of a plurality of possible hand gestures as determined from a position of the respective laser detector, wherein respective confusion matrices of the plurality of laser detectors collectively form a set of confusion matrices; and

determining, using the controller, a hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices.

19. The method according to claim 18 , wherein determining the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices comprises weighting each respective distance by its corresponding confusion matrix.

20. The method according to claim 18 , wherein determining the hand gesture from among the plurality of possible hand gestures based on the set of confusion matrices comprises:

determining, for each respective laser detector, a respective probable hand gesture based on the respective confusion matrix of the respective laser detector, wherein the respective probable hand gestures collectively form a set of probable hand gestures; and

selecting the hand gesture from the set of probable hand gestures based on a majority vote.

Assignments (1)
CHANGE OF NAME Recorded Dec 8, 2023
From: STMICROELECTRONICS SA
To: STMICROELECTRONICS FRANCE
Reel/Frame 065835/0159 →
Continuity (3)
Continuation 14863493 · Sep 24, 2015
Continuation 14863585 · Sep 24, 2015
Related Publication 20180136735A1 · May 17, 2018