Methods and systems of spatiotemporal pattern recognition for video content development
A media system generally includes a memory device that stores an event datastore that stores a plurality of event records, each event record corresponding to a respective event and event metadata describing at least one feature of the event. The media system (a) receives a request to generate an aggregated clip comprised of one or more media segments, where each media segment depicts a respective event; (b) for each event record from at least a subset of the plurality of event records, determines an interest level of the event corresponding to the event record; (c) determines one or more events to depict in the aggregated clip based on the respective interest levels of the one or more events; (d) generates the aggregated clip based on the respective media segments that depict the one or more events; and (e) transmits the aggregated clip to a user device.
1. A method comprising:
receiving a first broadcast video teed capturing a filmed occurrence, the first broadcast video feed comprising a plurality of broadcast video frames captured by a broadcast camera, wherein the first broadcast video teed is a video feed that is consumable by a client device;
receiving a tracking camera video feed corresponding to the filmed occurrence, the tracking camera video feed comprising a plurality of tracking video frames and being captured by a tracking camera having a position that is calibrated to a frame of reference;
tracking one or more respective pixel locations of an object detected in one or more respective broadcast video frames of the first broadcast video feed;
tracking one or more respective spatial locations of the object based on one or more respective tracking video frames where the object is detected in the tracking video feed;
time-aligning the first broadcast video feed with the tracking video feed based on the one or more respective pixel locations and the one or more respective spatial locations;
generating a spatio-temporal index corresponding to the filmed occurrence based on the time-alignment of the first broadcast video feed with the tracking video feed, wherein the spatio-temporal index indexes spatio-temporal information relating to objects detected in the broadcast video feed and/or the tracking video feed;
spatially aligning an augmentation item with respect to the object in a subset of the one or more broadcast video frames based on the spatio-temporal index; and
generating an augmented video stream having one or more augmented video frames based on the subset of the one or more broadcast video frames and the spatial alignment of the augmentation item with respect to the object, wherein the augmentation item and the object are spatially aligned in the augmented video stream.
2. The method of claim 1 , wherein the filmed occurrence is a sporting competition, and the object is a participant in the sporting competition and the augmentation item is a statistic relating to the participant that is obtained from a data feed corresponding to the sporting competition that is time aligned to the broadcast video feed.
3. The method of claim 1 , further comprising:
associating an advertisement with a type of event that is detectable in the subset of broadcast video frames; and
wherein generating the augmented video stream further comprises:
detecting an event depicted in a set of broadcast video frames of the broadcast video feed that is of the type of event associated with the advertisement; and
in response to detecting the event, augmenting at least one broadcast video frame with the advertisement.
4. The method of claim 1 , wherein the augmentation item is an advertisement, and the advertisement is spatially associated with the object that is detected in the subset of broadcast video frames.
5. The method of claim 1 , further comprising transmitting the augmented video stream to a client device.