Athlete style recognition system and method
View Patent ↗A system and method leverages understanding of complex dribbling video clips by representing a video sequence with a single Dribble Energy Image (DEI) that is informative for dribbling styles recognition. To overcome the shortage of labelled data, a dataset of soccer video clips employs Mask-RCNN to segment out dribbling players and OpenPose to obtain joints information of dribbling players. To solve issues caused by camera motions in highlight soccer videos, the system registers a video sequence to generate a single image representation DEI and dribbling styles classification.
1. A system for athletic style recognition, comprising:
an assembled collection of video clips that illustrate different players and play skills;
an affine-transformation-based module capable of registering a sequence of frames with target players performing a target skill into a single image representation;
a neural network trained to classify the target skill;
wherein the target skill is dribbling, and the target players are soccer players; and
a dribble energy image (DEI) configured to transfer a sequence of frames to an image representation using affine-transformation-based image registration.
2. The system of claim 1 , wherein in the assembled collection of video clips are of real athletic play.
3. The system of claim 1 , wherein the assembled collection of video clips is of simulated athletic play.
4. A system for athletic style recognition, comprising:
an assembled collection of video clips that illustrate different players and play skills;
an affine-transformation-based module capable of registering a sequence of frames with target players performing a target skill into a single image representation;
a neural network trained to classify the target skill;
wherein the neural network comprises a conditional generative adversarial network (GAN); and
a module for constructing a dribbling player's joints model as probability conditions for training the conditional GAN to generate a dribble energy image (DEI) wherein objects are guided to follow an embedding of a soccer player's body.
5. The system of claim 4 , comprising
a module configured to train the neural network on team dependent and team independent datasets.
6. The system of claim 5 , further comprising a real-time object detection module configured to predict multiple bounding boxes for an image along with respective class probabilities for each bounding box using YOLO9000.
7. The system of claim 5 , wherein the network is trained on the PASCAL VOC 2007 dataset.
8. The system of claim 5 , comprising a module for after detecting soccer players in consecutive frames using DeepSort tracking to track the players over consecutive frames and to formulate associations of the soccer players.
9. The system of claim 5 , further comprising a module for performing histogram matching for identifying the team of a player.
10. The system of claim 5 , further comprising a module for augmenting the datasets using DCGAN.
11. A method for athletic style recognition, comprising:
assembling a collection of video clips that illustrate different players and play skills;
registering, using an affine-transformation-based module, a sequence of frames with target players performing a target skill into a single image representation; and
training a neural network to classify the target skill;
wherein the target skill is dribbling, and the target players are soccer players; and
providing a dribble energy image (DEI) configured to transfer a sequence of frames to an image representation using affine-transformation-based image registration.
12. The method of claim 11 , wherein in the collection of video clips are of real athletic play.
13. The method of claim 11 , wherein the collection of video clips is of simulated athletic play.
14. A method for athletic style recognition, comprising:
assembling a collection of video clips that illustrate different players and play skills;
registering, using an affine-transformation-based module, a sequence of frames with target players performing a target skill into a single image representation; and
training a neural network to classify the target skill;
wherein the target skill is dribbling, and the target players are soccer players; and
wherein the neural network comprises a conditional generative adversarial network (GAN); and
further comprising constructing a dribbling player's joints model as probability conditions for training the conditional GAN to generate a dribble energy image (DEI) wherein objects are guided to follow an embedding of a soccer player's body.
15. The method of claim 14 , comprising
training the neural network on team dependent and team independent datasets.
16. The method of claim 15 , further comprising providing a real-time object detection module configured to predict multiple bounding boxes for an image along with respective class probabilities for each bounding box using YOLO9000.
17. The method of claim 15 , wherein the network is trained on the PASCAL VOC 2007 dataset.
18. The method of claim 15 , comprising a after detecting soccer players in consecutive frames using DeepSort tracking to track the players over consecutive frames and to formulate associations of the soccer players.
19. The method of claim 15 , further comprising performing histogram matching for identifying the team of a player.
20. The method of claim 15 , further comprising augmenting the datasets using DCGAN.