System and method for generating player tracking data from broadcast video
A system and method of generating a player tracking prediction are described herein. A computing system retrieves a broadcast video feed for a sporting event. The computing system segments the broadcast video feed into a unified view. The computing system generates a plurality of data sets based on the plurality of trackable frames. The computing system calibrates a camera associated with each trackable frame based on the body pose information. The computing system generates a plurality of sets of short tracklets based on the plurality of trackable frames and the body pose information. The computing system connects each set of short tracklets by generating a motion field vector for each player in the plurality of trackable frames. The computing system predicts a future motion of a player based on the player's motion field vector using a neural network.
1. A method, comprising:
identifying, by a computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting, by the computing system, the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating, by the computing system, body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing, by the computing system, future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.
2. The method of claim 1 , wherein segmenting, by the computing system, the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.
3. The method of claim 1 , further comprising:
identifying, by the computing system, a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.
4. The method of claim 3 , further comprising:
generating, by the computing system, a motion field vector for each player in the plurality of trackable frames.
5. The method of claim 4 , wherein constructing, by the computing system, the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.
6. A system for generating a player tracking prediction, comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform one or more operations comprising:
identifying a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.
7. The system of claim 6 , wherein segmenting the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.
8. The system of claim 6 , wherein the one or more operations further comprise:
identifying a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.
9. The system of claim 8 , wherein the one or more operations further comprise:
generating a motion field vector for each player in the plurality of trackable frames.
10. The system of claim 9 , wherein constructing the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.
11. A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors, causes a computing system to perform one or more operations comprising:
identifying, by the computing system, a broadcast video feed for a sporting event, the broadcast video feed comprising a plurality of video frames;
segmenting, by the computing system, the broadcast video feed into a unified view, wherein the unified view comprises a plurality of trackable frames, the plurality of trackable frames is a subset of the plurality of video frames;
generating, by the computing system, body pose information for each player in each trackable frame of the plurality of trackable frames; and
constructing, by the computing system, future motion of a player based on the plurality of trackable frames and the body pose information, the constructing comprising:
projecting motion of the player when the player has left a field of view and is not visible in the broadcast video feed, the projecting comprising:
identifying a first set of frames in which the player is present,
identifying a second set of frames following the first set of frames in which the player is not present, and
predicting a trajectory of the player based on prior trajectories of the player in the first set of frames.
12. The non-transitory computer readable medium of claim 11 , wherein segmenting, by the computing system, the broadcast video feed into the unified view comprises:
parsing the broadcast video feed to identify a first subset of video frames corresponding to a same view of the sporting event; and
discarding a second subset of video frames corresponding to a different view of the sporting event.
13. The non-transitory computer readable medium of claim 11 , further comprising:
identifying, by the computing system, a pattern of motion between two successive trackable frames by identifying players in each frame using the body pose information.
14. The non-transitory computer readable medium of claim 13 , further comprising:
generating, by the computing system, a motion field vector for each player in the plurality of trackable frames.
15. The non-transitory computer readable medium of claim 14 , wherein constructing, by the computing system, the future motion of the player based on the plurality of trackable frames and the body pose information comprises:
generating, via a neural network, the future motion of the player based on the motion field vector generated for the player.