IP Library Granted Patent US 10,930,172
Granted Patent B2
US 10,930,172 · App. 16/792,190 · Granted Feb 23, 2021

Methods and systems for facilitating interactive training of body-eye coordination and reaction time

Inventors: Qi Zhang (Hong Kong, HK); Arron Mollet (San Jose, CA); Wing Hung Chan (Hong Kong, HK); Keng Fai Lee (Cupertino, CA)
Assignee: NEX Team Inc.
G09B19/0038A63B24/0062A63B24/0075A63B71/0622A63B71/0669G06K9/00744G06K9/6256G06N3/08A61B5/1128A63B69/0071A63B2071/063A63B2071/0694A63B2220/806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,930,172
App. No.
16/792,190
Granted
Feb 23, 2021
Kind
B2
Abstract

Methods and systems for virtual coaching and performance training using a mobile device are disclosed. The methods and systems perform the steps of capturing a training video of one or more players, using the camera on the mobile computing device; superimposing a visual cue onto the training video at a first location and for a cue period starting from a first time instant; determining whether at least one of the one or more players has responded to the visual cue at a second time instant within the cue period, by analyzing a body posture flow of each player between the first time instant and the second time instant, wherein each body posture flow is extracted from the training video by performing a computer vision algorithm on one or more frames of the training video; and in response to determining that at least one player has responded to the visual cue, generating a feedback to the one or more players.

Claims (42)

1. A computer implemented method for facilitating training using a mobile computing device having a camera, the method comprising:

capturing a training video of one or more players using the camera on the mobile computing device;

superimposing a visual cue onto the training video at a first location and for a cue period starting from a first time instant;

determining whether at least one of the one or more players has responded to the visual cue at a second time instant within the cue period, by analyzing a body posture flow of each player between the first time instant and the second time instant, wherein each body posture flow is extracted from the training video by performing a computer vision algorithm on one or more frames of the training video; and

in response to determining that at least one player has responded to the visual cue, generating a feedback to the one or more players.

2. The computer implemented method of claim 1 , further comprising:

generating the feedback based on an identity of the at least one player that responded.

3. The computer implemented method of claim 1 , wherein the one or more players comprise at least two players.

4. The computer implemented method of claim 3 , wherein the extracting of at least two body posture flows from the training video comprises:

determining a plurality of player postures from the one or more frames of the training video; and

clustering the plurality of player postures into the at least two body posture flows.

5. The computer implemented method of claim 1 , wherein the visual cue is a symbol superimposed onto the training video at the first location of an image plane of the training video.

6. The computer implemented method of claim 5 , wherein the determining whether the at least one player has responded to the visual cue comprises determining a player movement to virtually touch the symbol in the image plane with a body part.

7. The computer implemented method of claim 5 , wherein the determining whether the at least one player has responded to the visual cue comprises determining whether the at least one player has virtually touched the symbol in the image plane with a sports equipment object.

8. The computer implemented method of claim 1 , wherein the determining whether the at least one player has responded to the visual cue comprises determining whether the at least one player has performed a predetermined sequence of movements.

9. The computer implemented method of claim 1 , further comprising:

determining a reaction time as a duration between the first time instant and the second time instant, wherein the feedback is a quality score generated based on the reaction time.

10. The computer implemented method of claim 1 , wherein the training comprises a plurality of difficulty levels based at least in part on a duration of the cue period.

11. The computer implemented method of claim 1 ,

wherein the computer vision algorithm comprises a Convolutional Neural Network (CNN) for detecting one or more key points of the player in an image plane, and

wherein the CNN module has been trained using one or more prior videos.

12. The computer implemented method of claim 1 ,

wherein the training video comprises a dribbling activity performed by the one or more players, and

wherein the superimposing the training video with the visual cue is in response to determining that the at least one player has dribbled for a predetermined number of times before the first time instant.

13. The computer implemented method of claim 1 , further comprising:

waiting for a period of wait time before the superimposing the training video with the visual cue, wherein a duration of the wait time is based on a detected player action during the wait time.

14. The computer implemented method of claim 1 ,

wherein the training video comprises a juggling activity performed by the one or more players, and

wherein the superimposing the training video with the visual cue is in response to determining that the at least one player has juggled for a predetermined number of times before the first time instant.

15. The computer implemented method of claim 1 , further comprising:

generating a training statistic for the at least one player based on the training video, wherein the training video comprises a dribbling activity performed by the one or more players, and wherein the training statistic includes at least one of a reaction speed and a dribbling speed.

16. The computer implemented method of claim 15 , wherein the training statistic comprises a first current statistic that is associated with the training video and a second historical statistic that is associated with one or more historical training sessions associated with the at least one player.

17. A computer implemented method for facilitating multi-player training using mobile computing devices each having a camera, the method comprising:

capturing a first training video of a first player using a first camera on a first mobile computing device;

superimposing a visual cue onto the first training video at a visual cue location and for a cue period starting from a first time instant;

determining whether the first player has responded to the visual cue at a second time instant within the cue period, by analyzing a body posture flow of the first player between the first time instant and the second time instant, wherein the body posture flow of the first player is extracted from the first training video by performing a computer vision algorithm on one or more frames of the first training video;

receiving a notification from a second mobile computing device, where the notification was generated in response to determining that a second player has responded to the visual cue at a third time instant within the cue period, by analyzing a body posture flow of the second player between the first time instant and the third time instant, wherein the body posture flow of the second player was extracted from a second training video of the second player by performing the computer vision algorithm on one or more frames of the second training video, wherein the second training video was captured using a second camera on the second mobile computing device, and wherein the visual cue was superimposed onto the second training video at the visual cue location starting from the first time instant; and

in response to determining that the first player has responded to the visual cue and to the notification, generating a feedback to the first player.

18. The computer implemented method of claim 17 , wherein the feedback is a score.

19. The computer implemented method of claim 17 , wherein the first training video and the second training video comprise basketball training activities.

20. The computer implemented method of claim 17 , further comprising:

receiving a second notification from a third mobile computing device, wherein the second notification was generated based on a third training video of a third player, captured using a third camera on the third mobile computing device.

Assignments (2)
SECURITY INTEREST Recorded Oct 9, 2025
From: NEX TEAM INC.
To: SILVER LAKE WATERMAN FUND III, L.P., AS AGENT
Reel/Frame 072524/0882 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2020
From: ZHANG, QI; MOLLETT, ARRON; CHAN, WING HUNG; LEE, KENG FAI
To: NEX TEAM INC.
Reel/Frame 051870/0456 →
Continuity (3)
Continuation 16555812 · Aug 29, 2019
Provisional Application 62778244 · Dec 11, 2018
Related Publication 20200184846A1 · Jun 11, 2020
Cited By (2)
US 12,194,357 US 12,508,485