SYSTEMS AND METHODS FOR SPORTS TRACKING USING DIFFUSION MODELS
A method for tracking one or more individuals during a sporting event, the method including: receiving, as an input, a geospatial data of a sporting event; receiving, as an input, labeled event data of the sporting event; performing multi-object tracking of one or more agents of the geospatial data to determine one or more vectors; inputting the labeled event data and one or more vectors into a diffusion model; and determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
1 . A computer implemented method for tracking one or more individuals during a sporting event, the method comprising:
receiving, as an input, geospatial data of a sporting event;
receiving, as an input, labeled event data based on sports broadcast footage of the sporting event;
performing multi-object tracking of one or more agents of the received geospatial data to determine one or more vectors;
inputting the labeled event data and one or more vectors into a diffusion model; and
determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
2 . The method of claim 1 , wherein the labeled event data includes a sequential stream of one or more major events throughout a sport event, the major events including at least one of a pass, shot, tackle, foul, turnover, penalty, goal, score, or substitution from the sporting event, and wherein the geospatial data includes one or more of the sports broadcast footage, in-venue footage, global positioning system (GPS) data, near field communication (NFC) data, or radio-frequency identification (RFID) data.
3 . The method of claim 1 , wherein the one or more vector includes at least one of an agent two dimensional coordinates on a sporting event's field, an agent position, an agent team, an indicator indicating the agent is a ball, or player visibility information.
4 . The method of claim 1 , wherein the event data is represented as a two dimensional spatiotemporal grid, the grid representing a stacking of each player's events.
5 . The method of claim 1 , wherein the diffusion model applies spatiotemporal axial attention on the received event data and one or more vectors, where self-attention is applied across temporal and spatial axis, separately.
6 . The method of claim 1 , wherein the diffusion model includes:
an event encoder; and
a tracking decoder,
wherein the event encoder encodes the labeled event data and the tracking decoder conditionally decodes trajectory sets.
7 . The method of claim 6 , wherein the event encoder embeds the event data, embedding the event data further comprising:
tokenzing the labeled event data using a linear projection;
applying sinusoidal positional embeddings to specify temporal occurrences of the event data;
processing the event data with stacked encoders; and
outputting event embeddings.
8 . The method of claim 7 , wherein the tracking decoder uses attention to embed and fuse the one or more vectors with the event embeddings.
9 . The method of claim 6 , further including:
a second tracking decoder; and
a transpose temporal convolution, the temporal convolution being configured to expand trajectories to their initial temporal dimensionality.
10 . A system for tracking one or more individuals during a sporting event, the system comprising:
a non-transitory computer readable medium configured to store processor-readable instructions; and
a processor operatively connected to the memory, and configured to execute the instructions to perform operations comprising:
receiving, as an input, geospatial data of a sporting event;
receiving, as an input, labeled event data of based on sports broadcast footage of the sporting event;
performing multi-object tracking of one or more agents of the received geospatial data to determine one or more vectors;
inputting the labeled event data and one or more vectors into a diffusion model; and
determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
11 . The system of claim 10 , wherein the labeled event data includes a sequential stream of one or more major events throughout a sport event, the major events including at least one of a pass, shot, tackle, foul, turnover, penalty, goal, score, or substitution from the sporting event, and wherein the geospatial data includes one or more of the sports broadcast footage, in-venue footage, global positioning system (GPS) data, near field communication (NFC) data, or radio-frequency identification (RFID) data.
12 . The system of claim 10 , wherein the one or more vector includes at least one of an agent two dimensional coordinates on a sporting event's field, an agent position, an agent team, an indicator indicating the agent is a ball, or player visibility information.
13 . The system of claim 10 , wherein the event data is represented as a two dimensional spatiotemporal grid, the grid representing a stacking of each player's events.
14 . The system of claim 10 , wherein the diffusion model applies spatiotemporal axial attention on the received event data and one or more vectors, where self-attention is applied across temporal and spatial axis, separately.
15 . The system of claim 10 , wherein the diffusion model includes:
an event encoder; and
a tracking decoder,
wherein the event encoder encodes the labeled event data and the tracking decoder conditionally decodes trajectory sets.
16 . The system of claim 15 , wherein the event encoder embeds the event data, embedding the event data further comprising:
tokenzing the labeled event data using a linear projection;
applying sinusoidal positional embeddings to specify temporal occurrences of the event data;
processing the event data with stacked encoders; and
outputting event embeddings.
17 . The system of claim 16 , wherein the tracking decoder uses attention to embed and fuse the one or more vectors with the event embeddings.
18 . The system of claim 15 , further including:
a second tracking decoder; and
a transpose temporal convolution, the temporal convolution being configured to expand trajectories to their initial temporal dimensionality.
19 . A non-transitory computer readable medium configured to store processor-readable instructions, wherein when executed by a processor, the instructions perform operations comprising:
receiving, as an input, geospatial data of a sporting event;
receiving, as an input, labeled event data based on sports broadcast footage of the sporting event;
performing multi-object tracking of one or more agents of the geospatial data to determine one or more vectors;
inputting the labeled event data and one or more vectors into a diffusion model; and
determining, using the diffusion model, one or more trajectory sequences for the one or more agents.
20 . The non-transitory computer readable medium of claim 19 , wherein the labeled event data includes a sequential stream of one or more major events throughout a sport event, the major events including at least one of a pass, shot, tackle, foul, turnover, penalty, goal, score, or substitution from the sporting event, and wherein the geospatial data includes one or more of the sports broadcast footage, in-venue footage, global positioning system (GPS) data, near field communication (NFC) data, or radio-frequency identification (RFID) data.