System for annotating media content for automatic content understanding
A method to correct for temporal variability in incoming streams of media and data to optimize the performance of a pattern recognition system includes the steps of receiving from one of the incoming streams a point in time when an event is announced, applying a probability distribution about the point in time, shifting a point of highest probability of the probability distribution back in time by an amount effective to accommodate for a delay between the event and the announcement, comparing a detected pattern of the event to a stored pattern of similar events and applying a confidence value to the comparison, and confirming to the pattern recognition system that the event occurred at the point of highest probability when the confidence score exceeds a predefined threshold. The method is useful to determine the time at which a particular play occurs during a sporting event, such as the time of a shot-on-goal in a soccer match.
1. A method to correct for temporal variability in incoming streams of media and data to optimize the performance of a pattern recognition system, comprising:
receiving from one of said incoming streams a point in time when an event is announced;
storing the point of time on a non-transitory media accessed by a processor programmed to transact the following remaining method steps;
applying a probability distribution about the point in time calculated with a digital processor and stored in memory;
shifting a point of highest probability of the probability distribution back in time by an amount effective to accommodate for a delay between the event and the announcement by copying stored values to a new address range that represents a shift in time;
comparing a detected pattern of the event to a stored pattern of similar events and applying a confidence value to the comparison; and
confirming to the pattern recognition system that the event occurred at the point of highest probability when the confidence score exceeds a predefined threshold.
2. The method of claim 1 wherein the detected pattern is generated by one or more sets of independently moving objects wherein each set has a common goal.
3. The method of claim 2 wherein the event occurs during a sporting event and a first set of independently moving objects is a first team and a second set of independently moving objects is an opposing second team.
4. The method of claim 3 wherein a player flow field is determined by mapping each member of each of the one or more sets based on location, direction of motion and velocity.
5. The method of claim 4 wherein the player flow field is determined from player location and changes in that location between adjacent frames of the incoming stream.
6. The method of claim 5 wherein the player flow field is compared to predefined player flow types to determine what play occurred and an outcome.
7. The method of claim 6 wherein the predefined player flow types are trained a priori using ground truth metadata.
8. The method of claim 2 wherein the pattern recognition system processes a live media stream and outputs player tracking metadata.
9. The method of claim 8 wherein the player tracking metadata is transformed into a player flow field.
10. The method of claim 9 wherein sequences of player flow fields are matched against predefined player flow types.
11. The method of claim 10 wherein a distance between PFF sequences is compared to a threshold that has been multiplied by the event probability (the augmented threshold).
12. The method of claim 10 including detecting the event if the distance exceeds an augmented threshold.