Automated system for providing video enhancements during sports broadcasts

1 . A computer-implemented method comprising:

receiving a first frame of tracking data indicating a respective location of each player of a first plurality of players on a field plane at a first time, wherein the tracking data is generated by a first plurality of sensors;

generating first embedding data representing a spatial formation of the first plurality of players at the first time based at least in part on the first frame of tracking data;

determining first game state data temporally associated with the first frame of tracking data, wherein the first game state data describes at least one of a current ball location, a current down, and a distance until a first down;

determining a subset of historical embeddings representing historical plays by filtering a set of historical plays using the first game state data;

determining, by searching the subset of historical embeddings using the first embedding data and a distance metric, a first set of historical plays;

determining, for each historical play of the first set of historical plays, a respective outcome of the respective historical play;

determining an area of the field plane corresponding to a highest concentration of successful outcomes among the respective outcomes of historical plays of the first set of historical plays; and

causing a first graphical overlay to be displayed in association with the area on a live video feed depicting the first plurality of players.

2 . The computer-implemented method of claim 1 , further comprising:

generating a Gaussian heatmap representing a first concentration of the successful outcomes in the area of the field plane and a second concentration of unsuccessful outcomes in a different area of the field plane, wherein the first graphical overlay comprises a visual representation of the Gaussian heatmap.

3 . The computer-implemented method of claim 1 , further comprising:

determining, for each of the historical plays of the first set of historical plays, an expected points added statistical value;

generating a respective two-dimensional data point in the field plane for each of the historical plays of the first set of historical plays;

generating weighted two-dimensional data points by weighting each of the two-dimensional data points using the corresponding expected points added statistical value; and

determining the area of the field plane corresponding to the highest concentration of successful outcomes based at least in part on the weighted two-dimensional data points.

4 . A computer-implemented method comprising:

receiving first tracking data representing first respective locations of a first plurality of players on a two-dimensional plane at a first time;

generating first embedding data representing a formation of the first plurality of players at the first time based at least in part on the first tracking data;

determining a first set of historical plays based at least in part on searching a first data store using the first embedding data, the first data store comprising a plurality of historical embeddings representing a plurality of past plays;

determining a first area of the two-dimensional plane corresponding to a play vulnerability based on respective outcomes of the first set of historical plays;

transforming the first area from the two-dimensional plane to a second area in a video plane of a live video feed using homography; and

causing a first graphical overlay to be displayed in association with the second area of the video plane on the live video feed.

5 . The computer-implemented method of claim 4 , further comprising:

generating first two-dimensional plot data comprising a respective data point for each of the plurality of past plays; and

generating, for each of the respective data points, a respective label associated with either a positive or negative outcome of the respective data point.

6 . The computer-implemented method of claim 4 , further comprising:

receiving, from a first metadata service, first game state data associated with the first tracking data; and

determining a subset of the plurality of past plays by filtering the plurality of past plays using the first game state data, wherein the determining the first set of historical plays comprises searching the subset of the plurality of past plays using the first embedding data.

7 . The computer-implemented method of claim 4 , further comprising:

determining a first subset of the first set of historical plays associated with a successful outcome of an offensive team;

determining a second subset of the first set of historical plays associated with an unsuccessful outcome of the offensive team; and

determining the first area based at least in part on a concentration of the first subset of the first set of historical plays in the first area.

8 . The computer-implemented method of claim 4 , further comprising:

receiving, from a first metadata service, first game state data associated with the first tracking data; and

generating, by a first encoder, the first embedding data further representing the first game state data.

9 . The computer-implemented method of claim 4 , further comprising:

receiving second tracking data representing second respective locations of the first plurality of players on the two-dimensional plane at a second time;

generating second embedding data representing the formation of the first plurality of players at the second time based at least in part on the second tracking data;

generating an aggregated embedding data based at least in part on a combination of the first embedding data and the second embedding data; and

determining a second set of historical plays based at least in part on searching the first data store using the aggregated embedding data.

10 . The computer-implemented method of claim 4 , wherein the first graphical overlay comprises a heat map indicating a concentration of successful outcomes for offensive teams associated with the first area.

11 . The computer-implemented method of claim 4 , wherein the first graphical overlay comprises a polygon indicating the first area associated with successful outcomes for offensive teams.

12 . The computer-implemented method of claim 4 , further comprising:

generating first two-dimensional plot data comprising a respective data point for each of the plurality of past plays, wherein each of the respective data points is associated with a successful outcome or an unsuccessful outcome for at least one offensive team; and

determining the first area based at least in part on determining a bounded area comprising a highest concentration of data points associated with successful outcomes.

13 . A system comprising:

at least one processor; and

non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to cause the at least one processor to:

receive first tracking data representing first respective locations of a first plurality of players on a two-dimensional plane at a first time;

generate first embedding data representing a formation of the first plurality of players at the first time based at least in part on the first tracking data;

determine a first set of historical plays based at least in part on searching a first data store using the first embedding data, the first data store comprising a plurality of historical embeddings representing a plurality of past plays;

determine a first area of the two-dimensional plane corresponding to a play vulnerability based on respective outcomes of the first set of historical plays;

transform the first area from the two-dimensional plane to a second area in a video plane of a live video feed using homography; and

cause a first graphical overlay to be displayed in association with the second area of the video plane on the live video feed.

14 . The system of claim 13 , wherein the non-transitory computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to:

generate first two-dimensional plot data comprising a respective data point for each of the plurality of past plays; and

generate, for each of the respective data points, a respective label associated with either a positive or negative outcome of the respective data point.

15 . The system of claim 13 , wherein the non-transitory computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to:

receive, from a first metadata service, first game state data associated with the first tracking data; and

determine a subset of the plurality of past plays by filtering the plurality of past plays using the first game state data, wherein the determining the first set of historical plays comprises searching the subset of the plurality of past plays using the first embedding data.

16 . The system of claim 13 , wherein the non-transitory computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to:

determine a first subset of the first set of historical plays associated with a successful outcome of an offensive team;

determine a second subset of the first set of historical plays associated with an unsuccessful outcome of the offensive team; and

determine the first area based at least in part on a concentration of the first subset of the first set of historical plays in the first area.

17 . The system of claim 13 , wherein the non-transitory computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to:

receive, from a first metadata service, first game state data associated with the first tracking data; and

generate, by a first encoder, the first embedding data further representing the first game state data.

18 . The system of claim 13 , wherein the non-transitory computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to:

receive second tracking data representing second respective locations of the first plurality of players on the two-dimensional plane at a second time;

generate second embedding data representing the formation of the first plurality of players at the second time based at least in part on the second tracking data;

generate an aggregated embedding data based at least in part on a combination of the first embedding data and the second embedding data; and

determine a second set of historical plays based at least in part on searching the first data store using the aggregated embedding data.