IP Library › Granted Patent US 9,996,109
Granted Patent B2
US 9,996,109 · App. 14/826,437 · Granted Jun 12, 2018

Identifying gestures using motion data

Inventors: Rodrigo Carceroni (Mountain View, CA); Derya Ozkan (Mountain View, CA); Suril Shah (Mountain View, CA); Pannag Raghunath Sanketi (Fremont, CA)
Assignee: Google LLC
G06F1/163G06F1/3206G06F1/3265G06F3/014G06F3/017G06F3/038G06F3/0346G06F2200/1637Y02D10/153
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,996,109
App. No.
14/826,437
Filed
Aug 14, 2015
Granted
Jun 12, 2018
Kind
B2
Art Unit
2694
USPC
345/156
Abstract

In one example, a method includes determining, by a processor ( 104 ) of a wearable computing device ( 102 ) and based on motion data generated by a motion sensor ( 106 ) of the wearable computing device, one or more strokes. In this example, the method also includes generating, by the processor and based on the motion data, a respective attribute vector for each respective stroke from the one or more strokes and classifying, by the processor and based on the respective attribute vector, each respective stroke from the one or more strokes into at least one category. In this example, the method also includes determining, by the processor and based on a gesture library and the at least one category for each stroke from the one or more strokes, a gesture. In this example, the method also includes performing, by the wearable device and based on the gesture, an action.

Claims (74)

1. A method comprising:

determining, by a processor of a wearable computing device and based on motion data generated by a motion sensor of the wearable computing device, one or more strokes, wherein the one or more strokes are determined by segmenting the motion data based on the features of the motion data;

generating, by the processor and based on the motion data, a respective attribute vector for each respective stroke from the one or more strokes;

classifying, by the processor and based on the respective attribute vector, each respective stroke from the one or more strokes into at least one category;

determining, by the processor and based on a gesture library and the at least one category for each stroke from the one or more strokes, a gesture; and

performing, by the wearable device and based on the gesture, an action.

2. The method of claim 1 , further comprising:

generating task-specific motion data by at least converting the motion data into a task-specific coordinate system; and

determining, based on a task-specific coordinate system and the task-specific motion data, an orientation of the wearable device relative to a gravity vector,

wherein determining the gesture is further based on the orientation of the wearable device.

3. The method of claim 2 , wherein:

the wearable computing device is operating in a first power mode while the motion sensor generates the motion data,

the orientation of the wearable device indicates that a user of the wearable device is viewing a display of the wearable device,

the gesture comprises a lift-and-look gesture, and

performing the action comprises transitioning, by the wearable device, a display of the wearable device from operating in the first power mode to operating in a second power mode, wherein the display uses less power while operating in the first power more than while operating in the second power mode.

4. The method of claim 1 , further comprising:

generating, by the motion sensor of the wearable computing device, the motion data;

responsive to determining, by a processor of the motion sensor that is different than the processor of the wearable computing device, that the motion data satisfies one or more thresholds, outputting the motion data to the processor of the wearable computing device.

5. The method of claim 4 , wherein the wearable computing device includes an application processor that is different than the processor of the wearable computing device and the processor of the motion sensor.

6. The method of claim 1 , wherein classifying each stroke from the plurality of stokes comprises classifying each stroke into a binary classification using support vector machines.

7. The method of claim 1 , wherein classifying each stroke from the plurality of strokes comprises:

classifying, by the processor and based on a distance function between feature vectors of each stroke from the plurality of strokes, each stroke from the plurality of strokes into the at least one category.

8. The method of claim 1 , wherein classifying each stroke from the plurality of strokes comprises:

classifying, by the processor and using deep belief networks, each stroke from the plurality of strokes.

9. The method of claim 1 , wherein the motion data includes a sequence of values, and wherein determining the one or more strokes comprises:

determining a local minimum value within a neighborhood of the sequence of values; and

responsive to determining that the local minimum value corresponds to a curvature of the motion data:

determining that the local minimum value is a starting boundary of a stroke of the one or more strokes;

evaluating subsequent values within the sequence of values to determine a value corresponding to an edge of the motion data; and

determining that the value corresponding to the edge is an ending boundary of the stroke.

10. The method of claim 9 , wherein the stroke is a first stroke of the one or more strokes, wherein the edge is a first edge, the method further comprising:

responsive to determining that the value that corresponds to the first edge is an ending boundary of the stroke:

evaluating subsequent values within the sequence of values to determine a value corresponding to a second edge of the motion data; and

determining that the value corresponding to the second edge is a starting boundary of a second stroke.

11. A wearable computing device comprising:

a motion sensor;

one or more processors configured to:

determine, based on motion data generated by a motion sensor of the wearable computing device, one or more strokes, wherein, to determine the one or more strokes, the one or more processors are configured to segment the motion data based on the features of the motion data;

generate, based on the motion data, a respective attribute vector for each respective stroke from the one or more strokes;

classify, based on the respective attribute vector, each respective stroke from the one or more strokes into at least one category;

determine, based on a gesture library and the at least one category for each stroke from the one or more strokes, a gesture; and

perform, based on the gesture, an action.

12. The wearable computing device of claim 11 , wherein the one or more processors are configured to:

generate task-specific motion data by at least converting the motion data into a task-specific coordinate system;

determine, based on a task-specific coordinate system and the task-specific motion data, an orientation of the wearable device relative to a gravity vector; and

determine the gesture further based on the orientation of the wearable device.

13. The wearable computing device of claim 12 , wherein:

the wearable computing device is operating in a first power mode while the motion sensor generates the motion data,

the orientation of the wearable device indicates that a user of the wearable device is viewing a display of the wearable device,

the gesture comprises a lift-and-look gesture, and

the one or more processors are configured to perform the action by at least transitioning, a display of the wearable device from operating in the first power mode to operating in a second power mode, wherein the display uses less power while operating in the first power more than while operating in the second power mode.

14. The wearable computing device of claim 11 , wherein the motion sensor is associated with a processor of the one or more processors, and wherein the processor associated with the motion sensor is configured to:

receive the motion data generated by the motion sensor of the wearable computing device; and

responsive to determining that the motion data satisfies one or more thresholds, output the motion data to another processor of the one or more processors of wearable computing device.

15. The wearable computing device of claim 14 , wherein the wearable computing device comprises an application processor that is different than the other processor of the wearable computing device and the processor associated with the motion sensor.

16. The wearable computing device of claim 11 , wherein the one or more processors are configured to classify each stroke from the plurality of stokes by at least classifying each stroke into a binary classification using support vector machines.

17. The wearable computing device of claim 11 , wherein the one or more processors are configured to classify each stroke from the plurality of strokes by at least:

classifying, based on a distance function between feature vectors of each stroke from the plurality of strokes, each stroke from the plurality of strokes into the at least one category.

18. The wearable computing device of claim 11 , wherein the motion data includes a sequence of values, and wherein the one or more processors are configured to determine the one or more motion strokes by at least:

determining a local minimum value within a neighborhood of the sequence of values; and

responsive to determining that the local minimum value corresponds to a curvature of the motion data:

determining that the local minimum value is a starting boundary of a stroke of the one or more strokes;

evaluating subsequent values within the sequence of values to determine a value corresponding to an edge of the motion data; and

determining that the value corresponding to the edge is an ending boundary of the stroke.

19. The wearable computing device of claim 18 , wherein the stroke is a first stroke of the one or more strokes, wherein the edge is a first edge, and wherein the one or more processors are configured to:

responsive to determining that the value that corresponds to the first edge is an ending boundary of the stroke:

evaluate subsequent values within the sequence of values to determine a value corresponding to a second edge of the motion data; and

determine that the value corresponding to the second edge is a starting boundary of a second stroke.

20. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor of a wearable computing device to:

determine, based on motion data generated by a motion sensor of the wearable computing device, one or more strokes, wherein the one or more strokes are determined by segmenting the motion data based on the features of the motion data;

generate, based on the motion data, a respective attribute vector for each respective stroke from the one or more strokes;

classify, based on the respective attribute vector, each respective stroke from the one or more strokes into at least one category;

determine, based on a gesture library and the at least one category for each stroke from the one or more strokes, a gesture; and

perform an action based on the gesture.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2015
From: CARCERONI, RODRIGO; OZKAN, DERYA; SHAH, SURIL; SANKETI, PANNAG RAGHUNATH
To: GOOGLE INC.
Reel/Frame 036612/0525 →
Continuity (2)
Provisional Application 62038229 · Aug 16, 2014
Related Publication 20160048161A1 · Feb 18, 2016