IP Library Granted Patent US 9,158,380
Granted Patent B2
US 9,158,380 · App. 13/544,930 · Granted Oct 13, 2015

Identifying a 3-D motion on 2-D planes

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,158,380
App. No.
13/544,930
Granted
Oct 13, 2015
Kind
B2
Abstract

Systems and methods for recognizing 3-D human motions are described. In some embodiments, the present disclosure contemplates a variety of methods and systems for recognizing a 3-D motion on 2-D planes. The 3-D human motions may include, but is not limited to, kinematic based gestures, such as clapping, spell casting, security gestures, etc.

Claims (49)

1. A computer-implemented method for recognizing a specific gesture based on a 3-D motion of a portable device, the computer-implemented method comprising:

obtaining linear acceleration motion data and device orientation data of the portable device in reference to gravity at a plurality of times at fixed interval, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by one or more accelerometers associated with the portable device;

obtaining relative angular rotation data of the portable device at the plurality of times, the angular rotation data including an angular velocity vector;

wherein the relative angular rotation data is obtained by one or more gyroscopes, or multiple depth cameras associated with the portable device;

defining a plurality of 2-D local planes at each of the plurality of times according to the linear acceleration vector, and the angular velocity vector obtained at each of the respective plurality of times;

analyzing 2-D kinematic data associated with the motion of the portable device on each of the plurality of defined 2-D local planes, the 2-D kinematic data obtained via the one or more accelerometers and the one or more gyroscopes or multiple depth cameras;

obtaining non-kinematic data of the portable device, the non-kinematic data including audio data from an audio input device associated with the portable device;

wherein the audio data is indicative of a predefined gesture;

identifying the 3-D motion by comparing the 2-D kinematic data associated with the motion of the portable device in one or more of the plurality of 2-D local planes;

identifying the specific gesture by analyzing the identified 3-D motion and the non-kinematic data of the portable device; and

performing or initiating a predefined action indicated by the identified specific gesture.

2. The computer-implemented method as recited in claim 1 , wherein the one or more accelerometers include a 3-axis accelerometer.

3. The computer-implemented method as recited in claim 1 , wherein the relative angular rotation data includes relative degree of rotation data on roll axis, pitch axis, and yaw axis.

4. The computer-implemented method as recited in claim 3 , wherein the one or more gyroscopes include a 3-axis gyroscope.

5. The computer-implemented method as recited in claim 3 , wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

6. The computer-implemented method as recited in claim 1 , further comprising: generating a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

7. The computer-implemented method as recited in claim 6 , wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator's 3-D motion pattern over time using predefined machine learning algorithms.

8. The computer-implemented method as recited in claim 7 , further comprising: storing the operator's 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

9. The computer-implemented method as recited in claim 1 , further comprising: providing an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator's physical characteristics.

10. The computer-implemented method as recited in claim 9 , wherein the portable device includes the audio input device.

11. The computer-implemented method as recited in claim 1 , wherein the portable device is a wireless controller, a personal digital assistant (PDA), a smart phone, a tablet, a Window/Mac laptop, a Linux machine, a netbook, a cloudbook, or a media player.

12. A system for recognizing a specific gesture based on a 3-D motion, the system comprising:

a portable device, the portable device including:

one or more processors;

one or more accelerometers;

one or more gyroscopes;

one or more audio input devices; and

one or more memory units with instructions stored thereon, which when executed by the one or more processors cause the system to:

obtain linear acceleration motion data and device orientation data of the portable device in reference to gravity at a plurality of times at fixed interval, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by the one or more accelerometers;

obtain relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

wherein the relative angular rotation data is obtained by the one or more gyroscopes;

define a plurality of 2-D local planes at each of the plurality of times according to the linear acceleration vector and the angular velocity vector;

analyze 2-D kinematic data associated with the motion of the portable device on each of the plurality of defined 2-D local planes, the 2-D kinematic data obtained via the one or more accelerometers and the one or more gyroscopes;

obtain non-kinematic data of the portable device, the non-kinematic data including audio data from the one or more audio input device associated with the portable device;

wherein, the audio input data is indicative of a predefined gesture;

identify the 3-D motion by comparing the 2-D kinematic data associated with the motion of the portable device in one or more of the plurality of 2-D local planes;

identify the specific gesture by analyzing the identified 3-D motion and the kinematic data of the portable device; and

perform or initiate a predefined action indicated by the identified specific gesture.

13. The system as recited in claim 12 , wherein the one or more accelerometers include a 3-axis accelerometer.

14. The system as recited in claim 12 , wherein the relative angular rotation data includes relative degree of rotation data on roll axis, pitch axis, and yaw axis.

15. The system as recited in claim 14 , wherein the one or more gyroscopes include a 3-axis gyroscope.

16. The system as recited in claim 14 , wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

17. The system as recited in claim 12 , wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further: generate a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

18. The system as recited in claim 17 , wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator's 3-D motion pattern over time using predefined machine learning algorithms.

19. The system as recited in claim 18 , wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further: store the operator's 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

20. The system as recited in claim 12 , wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further: provide an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator's physical characteristics.

21. The system for recognizing a specific gesture based on a 3-D motion as recited in claim 12 , wherein the portable device is a wireless controller, a personal digital assistant (PDA), a smart phone, a tablet, or a laptop.

Assignments (15)
CORRECTIVE ASSIGNMENT TO CORRECT THE ADDRESS FROM 71 S. WACKER DR. SUITE 2385 CHICAGO, ILLINOIS 60606 TO 459 18TH ST., #313, SAN FRANCISCO, CALIFORNIA 94107 PREVIOUSLY RECORDED ON REEL 039810 FRAME 0277. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Oct 24, 2016
From: NET POWER AND LIGHT, INC.
To: WICKR INC.
Reel/Frame 040267/0459 →
MERGER Recorded Aug 25, 2016
From: NET POWER AND LIGHT, INC.
To: WICKR INC.
Reel/Frame 039810/0277 →
MERGER Recorded Aug 24, 2016
From: NET POWER AND LIGHT, INC.
To: NET POWER AND LIGHT, INC.
Reel/Frame 039809/0346 →
RELEASE OF SECURITY INTEREST Recorded Apr 28, 2016
From: NET POWER & LIGHT, INC.
To: NET POWER & LIGHT, INC.
Reel/Frame 038398/0958 →
NOTE AND WARRANT CONVERSION AGREEMENT Recorded Apr 28, 2016
From: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOV8 PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHIN, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUI TY
To: NET POWER & LIGHT, INC.
Reel/Frame 038543/0966 →
NOTE AND WARRANT CONVERSION AGREEMENT Recorded Apr 28, 2016
From: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOV8 PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHIN, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUI TY
To: NET POWER & LIGHT, INC.
Reel/Frame 038543/0942 →
RELEASE OF SECURITY INTEREST Recorded Apr 28, 2016
From: NET POWER & LIGHT, INC.
To: NET POWER & LIGHT, INC.
Reel/Frame 038398/0946 →
NOTE AND WARRANT CONVERSION AGREEMENT Recorded Apr 27, 2016
From: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.
To: NET POWER & LIGHT, INC.
Reel/Frame 038543/0839 →
RELEASE OF SECURITY INTEREST Recorded Apr 27, 2016
From: NET POWER & LIGHT, INC.
To: NET POWER & LIGHT, INC.
Reel/Frame 038543/0831 →
SECURITY INTEREST Recorded Feb 22, 2016
From: NET POWER AND LIGHT, INC.
To: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOV8 PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHINE, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUI TY
Reel/Frame 037786/0090 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S NAME PREVIOUSLY RECORDED AT REEL: 037333 FRAME: 0680. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2015
From: NET POWER AND LIGHT, INC.
To: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOV8 PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHIN, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUITY
Reel/Frame 037394/0150 →
SECURITY INTEREST Recorded Dec 18, 2015
From: NET POWER AND LIGHT, INC.
To: PENINSULA TECHNOLOGY VENTURES, L.P.; PENSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOVS PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHIN, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUITY
Reel/Frame 037333/0680 →
SECURITY INTEREST Recorded Oct 6, 2015
From: NET POWER & LIGHT, INC.
To: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL 1, L.P.; SINGTEL INNOV8 PTE. LTD.; TWB INVESTMENT PARTNERSHIP II, LP; THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY (DAPER I); LOW, LAWRENCE B.; BROWN, JOHN SEELY; SHIN, JEANNIE; ORRICK INVESTMENTS 2010, LLC; ORRICK INVESTMENTS 2011, LLC; WANG, TA-HUI TY
Reel/Frame 036740/0252 →
SECURITY INTEREST Recorded Jun 4, 2014
From: NET POWER AND LIGHT, INC.
To: PENINSULA TECHNOLOGY VENTURES, L.P.; PENINSULA VENTURE PRINCIPALS, L.P.; ALSOP LOUIE CAPITAL I, L.P.
Reel/Frame 033086/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2012
From: DUBOVIK, GLEB
To: NET POWER AND LIGHT, INC.
Reel/Frame 028903/0434 →