IP Library › Granted Patent US 10,729,964
Granted Patent B2
US 10,729,964 · App. 16/697,376 · Granted Aug 4, 2020

Action detection and activity classification

Inventors: SantoshKumar Balakrishnan (Hillsboro, OR); Jordan M. Rice (Portland, OR); Steven H. Walker (Camas, WA); Adam Carroll (Bend, OR); Corey Dow-Hygelund (Sunriver, OR); Aaron K. Goodwin (Bend, OR); James M. Mullin (Bend, OR); Tye L. Rattenbury (New York, NY); Joshua M. Rooke-Ley (New York, NY); John M. Schmitt (Bend, OR)
Assignee: NIKE, Inc.
A63B71/06A41D1/002A43B3/0005A61B5/0002A61B5/0004A61B5/0022A61B5/0059A61B5/0077A61B5/01A61B5/02438A61B5/1112A61B5/1118A61B5/1123A61B5/486A61B5/681A61B5/6807A61B5/7246A61B5/7257A61B5/7264A61B5/7282A61B5/742A61B5/7475A63B24/0003A63B24/0062A63B24/0075G09B5/00G09B5/02G09B5/125G09B19/003G09B19/0038G16H20/30G16H20/40G16H40/67A61B2503/10A61B2562/029A61B2562/0219A61B2562/0252A61B2562/08A63B2024/0065
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,729,964
App. No.
16/697,376
Granted
Aug 4, 2020
Kind
B2
Abstract

Activities, actions and events during user performance of physical activity may be detected using various algorithms and templates. Templates may include an arrangement of one or more states that may identify particular event types and timing between events. Templates may be specific to a particular type of activity (e.g., types of sports, drills, events, etc.), user, terrain, time of day and the like.

Claims (61)

1. A computer-implemented method comprising:

receiving, by a sensor system having at least a processor and a memory, raw sensor data associated with athletic activity of a user, the raw sensor data being received from one or more sensing devices;

determining, by the sensor system and based on a signature of a signal received from the one or more sensing devices, a category of athletic activity being performed;

selecting, by an activity processing system, an athletic activity template for comparison, the athletic activity template being selected based on the category of athletic activity being performed;

comparing, by the activity processing system, one or more events in the raw sensor data to the selected of athletic activity template;

determining, by the activity processing system, whether the one or more events match the selected activity template, wherein the determining includes evaluating whether a first candidate athletic activity meets one or more relative constraints of a first state, wherein the one or more relative constraints defines a required relationship between the first candidate athletic activity and a second candidate athletic activity; and

responsive to determining that the one or more events match one or more activity templates, classifying the one or more events of the user as performance of a first action of a first type.

2. The computer-implemented method of claim 1 , further including:

determining, by the activity processing system, a quality of the first action, wherein determining the quality of the first action includes weighting at least one portion of the one or more matching activity templates more than at least one other portion of the one or more matching activity templates;

determining, by the activity processing system, whether the quality of the first action is below a predetermined threshold; and

responsive to determining that the quality of the first action is below the predetermined threshold, providing, by the activity processing system, coaching information to the user.

3. The computer-implemented method of claim 2 , wherein the coaching information includes at least a coaching instruction related to heel weight.

4. The computer-implemented method of claim 2 , wherein providing coaching information to the user further includes providing recommendations for improving the quality of the first type of action.

5. The computer-implemented method of claim 2 , wherein the coaching information provided to the user is specific to at least one of the one or more events.

6. The computer-implemented method of claim 2 , wherein the coaching information is generated based, at least in part, on data associated with weather and terrain at a location of performance of the athletic activity.

7. The computer-implemented method of claim 2 , wherein determining the quality of the first action further includes determining a strength of a match between the one or more events and the one or more activity templates.

8. The computer-implemented method of claim 1 , wherein at least one sensor of the one or more sensing devices is arranged in an article of footwear of the user.

9. One or more non-transitory computer-readable media storing computer readable instructions that, when executed, cause at least one computing device to:

determine a type of physical activity associated with an athletic activity performed by a user;

based on the determined type of physical activity performed by the user, identify an activity template associated with the physical activity;

receive raw sensor data associated with the athletic activity performed by the user, the raw sensor data being received from one or more sensing devices;

determine, based on a signature of a signal received from the one or more sensing devices, a category of athletic activity being performed;

select an athletic activity template for comparison, the athletic activity template being selected based on the determined category of athletic activity being performed;

compare one or more events in the raw sensor data to the selected activity template;

determine whether the one or more events the selected activity template, and evaluate whether a first candidate athletic activity meets one or more relative constraints of a first state, wherein the one or more relative constraints defines a required relationship between the first candidate athletic activity and a second candidate athletic activity; and

responsive to determining that the one or more events match one or more activity templates, classify the one or more events as performance of a first action of a first type.

10. The one or more non-transitory computer-readable media of claim 9 , further including instructions that, when executed, cause the computing device to:

determine a quality of the first action, wherein determining the quality of the first action further includes instructions that, when executed, cause the computing device to weight at least one portion of the one or more matching activity templates more than at least one other portion of the one or more matching activity templates;

determine whether the quality of the first action is below a predetermined threshold; and

responsive to determining that the quality of the first action is below the predetermined threshold, provide coaching information to the user.

11. The one or more non-transitory computer-readable media of claim 10 , wherein providing coaching information to the user further includes instructions that, when executed, cause the computing device to:

provide recommendations for improving the quality of the first type of action.

12. The one or more non-transitory computer-readable media of claim 10 , wherein the coaching information provided to the user is specific to at least one of the one or more events.

13. The one or more non-transitory computer-readable media of claim 9 , further including instructions that, when executed, cause the computer device to:

responsive to determining that the one or more events do not match one or more activity templates, storing the received sensor data.

14. The one or more non-transitory computer-readable media of claim 10 , wherein determining the quality of the first action further includes instructions that, when executed, cause the computing device to:

determine a strength of a match between the one or more events and the one or more activity templates.

15. The one or more non-transitory computer-readable media of claim 9 , wherein at least one sensor of the one or more sensing devices is arranged in an article of footwear of the user.

16. A system comprising:

a sensor system including a plurality of sensors, the sensor system further including:

a first processor; and

memory storing computer readable instructions that, when executed, cause the first processor to:

receive raw sensor data associated with athletic activity of a user;

determine, based on a signature of a signal received from the plurality of sensors, a category of athletic activity being performed; and

transmit, to an activity processing system, the determined category of athletic activity being performed and the raw sensor data; and

the activity processing system, including:

a second processor; and

memory storing computer-readable instructions that, when executed, cause the second processor to:

receive the transmitted determined category of athletic activity being performed;

select an athletic activity template for comparison, the athletic activity template being selected based on the determined category of athletic activity being performed;

compare one or more events detected in the raw sensor data to the activity template;

determine whether the one or more events match the selected activity template, and evaluating whether a first candidate athletic activity meets one or more relative constraints of a first state, wherein the one or more relative constraints defines a required relationship between the first candidate athletic activity and a second candidate athletic activity;

responsive to determining that the one or more events match the activity template, classify the one or more events of the user as performance of a first action of a first type.

17. The system of claim 16 , further including instructions that, when executed, cause the second processor to:

determine a quality of the first action, wherein determining the quality of the first action includes instructions that, when executed, cause the second processor to weight at least one portion of the one or more matching activity templates more than at least one other portion of the one or more matching activity templates;

determine whether the quality of the first action is below a predetermined threshold; and

responsive to determining that the quality of the first action is below the predetermined threshold, provide coaching information to the user.

18. The system of claim 17 , wherein determining the quality of the first action further includes instructions that, when executed, cause the second processor to:

determine a strength of a match between the one or more events and the one or more activity templates.

19. The system of claim 16 , wherein at least one sensor of the sensor system is arranged in an article of footwear of the user.

20. The system of claim 17 , wherein the coaching information provided to the user is specific to at least one of the one or more events.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: BALAKRISHNAN, SANTOSHKUMAR; RICE, JORDAN M.; WALKER, STEVEN H.
To: NIKE, INC.
Reel/Frame 053101/0884 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: RATTENBURY, TYE L.; ROOKE-LEY, JOSHUA M.
To: R/GA
Reel/Frame 053101/0920 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: R/GA
To: NIKE, INC.
Reel/Frame 053101/0929 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: CARROLL, ADAM; DOW-HYGELUND, COREY; GOODWIN, AARON K.; MULLIN, JAMES M.; SCHMITT, JOHN M.
To: BEND RESEARCH, INC.
Reel/Frame 053101/0954 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: BEND RESEARCH, INC.
To: NIKE, INC.
Reel/Frame 053101/0986 →
Continuity (7)
Continuation 16508553 · Jul 11, 2019
Continuation 15254548 · Sep 1, 2016
Continuation 15060899 · Mar 4, 2016
Continuation 14571664 · Dec 16, 2014
Continuation 13401592 · Feb 21, 2012
Provisional Application 61588608 · Jan 19, 2012
Related Publication 20200171371A1 · Jun 4, 2020
Cited By (2)
US 12,396,545 US 12,593,908