System and method for player reidentification in broadcast video
A system and method of re-identifying players in a broadcast video feed are provided herein. A computing system retrieves a broadcast video feed for a sporting event. The broadcast video feed includes a plurality of video frames. The computing system generates a plurality of tracks based on the plurality of video frames. Each track includes a plurality of image patches associated with at least one player. Each image patch of the plurality of image patches is a subset of the corresponding frame of the plurality of video frames. For each track, the computing system generates a gallery of image patches. A jersey number of each player is visible in each image patch of the gallery. The computing system matches, via a convolutional autoencoder, tracks across galleries. The computing system measures, via a neural network, a similarity score for each matched track and associates two tracks based on the measured similarity.
1. A method, comprising:
identifying, by a computing system, a plurality of tracks corresponding to a plurality of players from a broadcast video feed captured by a broadcast camera;
determining, by the computing system, one or more features in the plurality of tracks;
determining, by the computing system, a similarity score between a first set of features of the one or more features in a first track of the plurality of tracks and a second set of features of the one or more features in a second track of the plurality of tracks;
comparing, by the computing system, the similarity score to a threshold to determine that the similarity score is higher than the threshold; and
in response to the comparing, determining, by the computing system, that a first player of the plurality of players corresponding to the first set of features and a second player of the plurality of players corresponding to the second set of features are a same player.
2. The method of claim 1 , wherein each of the plurality of tracks include a player identity label, pose information, and a plurality of image patches.
3. The method of claim 2 , the method further comprising:
calculating, by the computing system, a plurality of similarity values between at least two image patches based on the one or more features; and
averaging, by the computing system, the plurality of similarity values to determine the similarity score.
4. The method of claim 3 , the method further comprising:
based on the determining the same player, associating, by the computing system, at least two tracks corresponding to the at least two image patches based on the one or more features.
5. The method of claim 1 , wherein the one or more features include a player clothing style, a player clothing color, a player skin color, or a player jersey number.
6. The method of claim 1 , wherein a trained machine-learning model is configured to perform the determining the one or more features in the plurality of tracks.
7. The method of claim 1 , wherein the first track comprises trajectory information corresponding to the first player up to a first time, wherein after the first time, the first player is not a field of view of a broadcast camera that captures the first track,
wherein the second track comprises second trajectory information corresponding to the second player up to a second time after the first time, and
wherein the first player is not in the field of view of the broadcast camera between the first time and the second time.
8. A computer system, the computer system comprising:
a memory having processor-readable instructions stored therein; and
one or more processors configured to access the memory and execute the processor-readable instructions, which when executed by the one or more processors configures the one or more processors to perform a plurality of functions, including functions for:
identifying a plurality of tracks corresponding to a plurality of players from a broadcast video feed captured by a broadcast camera;
determining one or more features in the plurality of tracks;
determining a similarity score between a first set of features of the one or more features in a first track of the plurality of tracks and a second set of features of the one or more features in a second track of the plurality of tracks;
comparing the similarity score to a threshold to determine that the similarity score is higher than the threshold; and
in response to the comparing, determining that a first player of the plurality of players corresponding to the first set of features and a second player of the plurality of players corresponding to the second set of features are a same player.
9. The computer system of claim 8 , wherein each of the plurality of tracks include a player identity label, pose information, and a plurality of image patches.
10. The computer system of claim 9 , the functions further comprising:
calculating a plurality of similarity values between at least two image patches based on the one or more features; and
averaging the plurality of similarity values to determine the similarity score.
11. The computer system of claim 10 , the functions further comprising:
based on the determining the same player, associating at least two tracks corresponding to the at least two image patches based on the one or more features.
12. The computer system of claim 8 , wherein the one or more features include a player clothing style, a player clothing color, a player skin color, or a player jersey number.
13. The computer system of claim 8 , wherein a trained machine-learning model is configured to perform the determining the one or more features in the plurality of tracks.
14. The computer system of claim 8 , wherein the first track comprises trajectory information corresponding to the first player up to a first time, wherein after the first time, the first player is not a field of view of a broadcast camera that captures the first track,
wherein the second track comprises second trajectory information corresponding to the second player up to a second time after the first time, and
wherein the first player is not in the field of view of the broadcast camera between the first time and the second time.
15. A non-transitory computer-readable medium storing instructions for causing a computing system to perform functions for generating a player tracking prediction, the functions comprising:
identifying, by a computing system, a plurality of tracks corresponding to a plurality of players from a broadcast video feed captured by a broadcast camera;
determining, by the computing system, one or more features in the plurality of tracks;
determining, by the computing system, a similarity score between a first set of features of the one or more features in a first track of the plurality of tracks and a second set of features of the one or more features in a second track of the plurality of tracks;
comparing, by the computing system, the similarity score to a threshold to determine that the similarity score is higher than the threshold; and
in response to the comparing, determining, by the computing system, that a first player of the plurality of players corresponding to the first set of features and a second player of the plurality of players corresponding to the second set of features are a same player.
16. The non-transitory computer-readable medium of claim 15 , wherein each of the plurality of tracks include a player identity label, pose information, and a plurality of image patches.
17. The non-transitory computer-readable medium of claim 16 , the instructions further comprising:
calculating, by the computing system, a plurality of similarity values between at least two image patches based on the one or more features; and
averaging, by the computing system, the plurality of similarity values to determine the similarity score.
18. The non-transitory computer-readable medium of claim 17 , the instructions further comprising:
based on the determining the same player, associating, by the computing system, at least two tracks corresponding to the at least two image patches based on the one or more features.
19. The non-transitory computer-readable medium of claim 15 , wherein the one or more features include a player clothing style, a player clothing color, a player skin color, or a player jersey number.
20. The non-transitory computer-readable medium of claim 15 , wherein a trained machine-learning model is configured to perform the determining the one or more features in the plurality of tracks.