Methods, systems, and user interface navigation of video content based spatiotemporal pattern recognition
A user interface for a media system supports using gestures, such as swiping gestures and taps, to navigate frame-synchronized video clips or video feeds. The detection of the gestures is interpreted as a command to navigate the frame-synchronized content. In one implementation, a tracking system and a trained machine learning system is used to generate the frame synchronized video clips or video feeds. In one implementation, video clips of an event are organized into storylines and the user interface permits navigation between different storylines and within individual storylines.
1. A method of navigating video content, comprising: generating a time-aligned video feed of an event and forming a set of frame-synchronized
video segments or frame-synchronized video feeds for the event; identifying related content of the event from the set of frame-synchronized video segments or video feeds;
detecting a swiping or tapping gesture of a user interface of a client device; and navigating a display of the set of (related) frame-synchronized video segments or frame-synchronized video feeds in response to the detected swiping or tapping gesture; and
wherein in response to a vertical swipe gesture, the navigating is toggled (toggles) between at least two different video camera angles of a selected video clip.
2. The method of claim 1 , further comprising organizing related content into a set of storylines with each storyline having a set of storyline elements, with each storyline element having an associated screen to display media content.
3. The method of claim 2 , wherein in response to a detected horizontal swiping gesture, navigating to a next or previous storyline.
4. The method of claim 2 , wherein in response to a detected tapping gesture, skipping to a next or previous element within a storyline.
5. The method of claim 2 , wherein in response to a detected vertical swiping gesture, accessing a different mode for an individual storyline element.
6. The method of claim 5 , wherein in response to detecting a vertical swipe gesture, the navigation is toggled to display (displays) a discover screen over a storyline element screen.
7. A method of navigating video content, comprising:
generating a time-aligned video feed of an event and forming a set of frame-synchronized video segments or video feeds for the event;
identifying related content of the event from the set of frame-synchronized video segments or frame-synchronized video feeds; and
detecting a swiping or tapping gesture of a user interface of a client device; and
navigating a display of the set of (related) frame-synchronized video segments or frame-synchronized video feeds in response to the detected gesture;
wherein in response to detecting a swiping gesture, the navigating switching the display between different frame-synchronized video streams of the same event from different camera angles, or different augmented versions of the video streams.
8. A method of navigating video content, comprising:
generating a time-aligned video feed of an event and forming a set of frame-synchronized video segments or frame-synchronized video feeds for the event;
identifying related content of the event from the set of frame-synchronized video segments or frame-synchronized video feeds;
detecting a swiping or tapping gesture of a user interface of a client device;
navigating a display of the set of frame-synchronized video segments or frame-synchronized video feeds in response to the detected swiping or tapping gesture; and
generating tracking information and a spatial and temporal index for time-aligning the video feeds.
9. The method of claim 7 , wherein generating tracking information includes time-aligning the (broadcast) video (feed) feeds with the tracking video feed based on (the) one or more respective pixel locations and (the) one or more respective spatial locations.
10. The method of claim 1 , wherein the gesture is a gesture entered on a touchscreen of a user client device.
11. A computer implemented method, comprising:
forming a set of frame-synchronized video segments or frame-synchronized video feeds for related content associated with an event using tracking information and a trained machine learning apparatus to perform spatial and temporal synchronization;
detecting a swiping or tapping gesture of a user interface of a client device; and navigating a display of the related content in response to the detected swiping or tapping gesture; and
wherein in response to detecting a swiping gesture, the navigating switching between different frame-synchronized streams of the same event from at least two different camera angles, or at least two different augmented versions of the video streams.
12. The method of claim 10 , wherein related content is organized into storylines with each storyline having at least one storyline element to display a video segment; and the navigating comprises navigating the storyline and storyline elements in response to the detected swiping or tapping gesture.
13. The method of claim 11 , wherein the detected swiping or tapping gesture comprises a horizontal swipe gesture, and navigating the set of related video segments comprises jumping to a next or previous storyline in response to detecting the horizontal swipe gesture.
14. The method of claim 11 , wherein the detected swiping or tapping gesture comprises a vertical swipe gesture and navigating the set of related video segments comprising toggling between different video clips of a set of storyline element screens.
15. The method of claim 11 , wherein the detected swiping or tapping gesture comprises a tap gesture and wherein the navigating comprises skipping to a next element within a storyline.
16. A computer implemented method, comprising:
forming a set of frame-synchronized video segments or frame-synchronized video feeds for related content associated with an event using tracking information and a trained machine learning apparatus to perform spatial and temporal synchronization;
detecting a swiping or tapping gesture of a user interface of a client device;
navigating a display of the related content in response to the detected swiping or tapping gesture;
receiving a tracking camera video feed comprising a plurality of tracking video frames being captured by a tracking camera having a position that is calibrated to a frame of reference;
tracking one or more respective pixel locations of an object detected in one or more respective broadcast video frames of a broadcast video feed;
tracking one or more respective spatial locations of the object based on one or more respective tracking video frames where the object is detected in the tracking camera video feed;
time-aligning the broadcast video feed with the tracking video feed based on the one or more respective pixel locations and the one or more respective spatial locations; and
generating a spatio-temporal index (corresponding to the filmed occurrence) based on the time-alignment of the broadcast video feed with the tracking video feed,
wherein the spatio-temporal index indexes spatio-temporal information relating to objects detected in the broadcast video feed and/or the tracking video feed.
17. The method of claim 10 , wherein the swiping or tapping gesture is a gesture entered on a touchscreen of a user client device.
18. A system, comprising:
a media system having a tracking system and a trained machine learning system trained to receive a video feed and generate a time-aligned video feed of an event, form a set of frame- synchronized video segments or frame-synchronized video feeds for the event, and organize related content of the event from the set of frame-synchronized video segments or frame-synchronized video feeds;
the media system including a gesture detection module to detect a swiping or tapping gesture of a user interface of a client device and in response navigate a display of the set of (related) frame-synchronized video segments or video feeds in response to the detected gesture; and
wherein in response to a vertical swipe gesture, the response to navigate the display includes toggling between at least two different video camera angles of the same video clip.
19. A system, comprising:
a media system having a tracking system and a trained machine learning system trained to receive a video feed and generate a time-aligned video feed of an event, form a set of frame- synchronized video segments or frame-synchronized video feeds
for the event, and organize related content of the event from the set of frame-synchronized video segments or frame-synchronized video feeds; the media system including a gesture detection module to detect a swiping or tapping
gesture of a user interface of a client device and in response navigate a display of the set of (related) frame-synchronized video segments or frame-synchronized video feeds in response to the detected swiping or tapping gesture; and wherein in response to detecting a swiping gesture, the navigating including switching
between different frame-synchronized streams of the same event from at least two different camera angles, or at least two different augmented versions of the video streams.
20. A system, comprising:
a media system having a tracking system and a trained machine learning system trained to receive a video feed, generate tracking information and a spatial and temporal index for time-aligning the video feed, and generate a time-aligned video feed of an event, form a set of frame-synchronized video segments or frame-synchronized video feeds for the event, and organize related content of the event from the set of frame-synchronized video segments or video feeds; and the media system including a gesture detection module to detect a swiping or tapping
gesture of a user interface of a client device and in response navigate a display of the set of (related) frame-synchronized video segments or frame-synchronized video feeds in response to the detected gesture.