Synthesizing a presentation of a multimedia event
View Patent ↗Example embodiments of a media synchronization system and method for synthesizing a presentation of a multimedia event are generally described herein. In some example embodiments, the media synchronization system includes a media ingestion module to access a plurality of media clips received from a plurality of client devices, a media analysis module to determine a temporal relation between a first media clip from the plurality of media clips and a second media clip from the plurality of media clips, and a content creation module to align the first media clip and the second media clip based on the temporal relation, and to combine the first media clip and the second media clip to generate the presentation.
1. A media synchronization system to synthesize a presentation of a multimedia event, the system comprising:
a media ingestion module configured to access a plurality of media clips received from a plurality of client devices that include a first client device from which a first media clip with a first audio waveform is received and a second client device from which a second media clip with a second audio waveform is received;
processing circuitry configured by a media analysis module to determine a temporal overlap of the first media clip received from the first client device with the second media clip received from the second client device, the temporal overlap of the first and second media clips being determined by matching a first fingerprint of at least part of the first media clip with a second fingerprint of at least part of the second media clip; and
a content creation module configured to merge the first media clip and the second media clip into a group of overlapping media clips based on the temporal overlap determined by the matching of the first fingerprint with the second fingerprint, and to generate the presentation to include the group formed by merging the first and second media clips based on the temporal overlap.
2. The system of claim 1 , the media ingestion module being configured to assign an identifier to the first media clip.
3. The system of claim 1 , the plurality of media clips comprising at least one of video data, audio data, image data, or text data.
4. The system of claim 1 , the plurality of media clips comprising a media clip recorded by a user.
5. The system of claim 1 , the media analysis module further configuring the processing circuitry to generate recognition data associated with the first media clip.
6. The system of claim 1 , the media analysis module further configuring the processing circuitry to group at least some of the plurality of media clips based on fingerprint matching.
7. The system of claim 1 , the first media clip comprising audio data associated with video data, the content creation module being configured to separately align the audio data and the video data.
8. The system of claim 7 , the content creation module being configured to separately align the audio data and the video data based on at least one of fingerprint matching, metadata matching, or lack of matching.
9. The system of claim 1 , the content creation module being configured to combine the first media clip and the second media clip based on at least one of automatic combining or user-supervised combining.
10. The system of claim 1 , the content creation module being configured to generate a media effect to be combined with the first media clip.
11. The system of claim 1 further comprising a content publishing module configured to publish the presentation.
12. The system of claim 11 , the content publishing module being further configured to publish the presentation to at least one of a mobile device, a stationary device, a public network, or a nonpublic network.
13. The system of claim 1 , wherein:
each of the plurality of media clips is generated from an event by the plurality of client devices; and
the media analysis module configures the processing circuitry to identify the first media clip and the second media clip as being generated from the event.
14. The system of claim 1 , wherein:
the first media clip and the second media clip are taken from the same event; and
the media analysis module configures the processing circuitry to compute the first fingerprint from the first media clip taken from the same event and compute the second fingerprint from the second media clip taken from the same event.
15. The system of claim 1 , wherein:
the first media clip and the second media clip are separately recorded views of a single event; and
the content creation module is configured to merge the first media clip and the second media clip based on a first audio fingerprint of the first media clip and on a second audio fingerprint of the second media clip.
16. The system of claim 1 , wherein:
the first media clip includes first footage filmed during a concert at a performance hall by the first client device located in a first part of the performance hall;
the second media clip includes second footage filmed during the concert at the performance hall by the second client device located in a second part of the performance hall.
17. The system of claim 1 , wherein:
the first media clip includes first footage of a concert at a performance hall filmed by the first client device from a first angle within the performance hall during a first period of time during the concert;
the second media clip includes second footage of the concert at the performance hall filmed by the second client device from a second angle within the performance hall during a second period of time that overlaps the first period of time during the concert.
18. A computerized method for synthesizing a presentation of a multimedia event, the method comprising:
accessing a plurality of media clips received from a plurality of client devices that include a first client device from which a first media clip with a first audio waveform is received and a second client device from which a second media clip with a second audio waveform is received;
determining a temporal overlap of the first media clip received from the first client device with the second media clip received from the second client device, the determining being performed by processing circuitry, the temporal overlap of the first and second media clips being determined by matching a first fingerprint of at least part of the first media clip with a second fingerprint of at least part of the second media clip;
merging the first media clip and the second media clip into a group of overlapping media clips based on the temporal overlap determined by the matching of the first fingerprint with the second fingerprint; and
generating the presentation to include the group formed by the merging of the first and second media clips based on the temporal overlap.
19. The method of claim 18 further comprising assigning an identifier to the first media clip.
20. The method of claim 18 , the plurality of media clips comprising at least one of video data, audio data, image data, or text data.
21. The method of claim 18 , the plurality of media clips comprising a media clip recorded by a user.
22. The method of claim 18 further comprising processing the first media clip to generate recognition data associated with the first media clip.
23. The method of claim 18 , the first media clip comprising audio data associated with video data, the merging comprising separately aligning the audio data and the video data.
24. The method of claim 23 , the separate aligning of the audio data and the video data comprising separately aligning the audio data and the video data based on at least one of fingerprint matching, metadata matching, or lack of matching.
25. The method of claim 18 , the combining comprising generating a media effect to be combined with the first media clip.
26. The method of claim 18 , further comprising publishing the presentation.
27. The method of claim 26 , the publishing comprising publishing to at least one of a mobile device, a stationary device, a public network, or a nonpublic network.
28. The method of claim 26 , the publishing comprising displaying an advertisement.
29. The method of claim 18 , wherein:
each of the plurality of media clips is generated from an event by the plurality of client devices; and
the determining of the temporal overlap of the first media clip and the second media clip includes identifying the first media clip and the second media clip as being generated from the event.
30. The method of claim 18 , wherein:
the first media clip and the second media clip are taken from the same event; and the method further comprises
computing the first fingerprint from the first media clip taken from the same event and computing the second fingerprint from the second media clip taken from the same event.
31. The method of claim 18 , wherein:
the first media clip and the second media clip are separately recorded views of a single event; and
the merging of the first media clip and the second media clip is based on a first audio fingerprint of the first media clip and on a second audio fingerprint of the second media clip.
32. The method of claim 18 , wherein:
the first media clip includes first footage filmed during a concert at a performance hall by the first client device located in a first part of the performance hall;
the second media clip includes second footage filmed during the concert at the performance hall by the second client device located in a second part of the performance hall.
33. The method of claim 18 , wherein:
the first media clip includes first footage of a concert at a performance hall filmed by the first client device from a first angle within the performance hall during a first period of time during the concert;
the second media clip includes second footage of the concert at the performance hall filmed by the second client device from a second angle within the performance hall during a second period of time that overlaps the first period of time during the concert.
34. A non-transitory machine-readable medium comprising instructions, which when implemented by one or more processors, cause the one or more processors to perform operations comprising:
accessing a plurality of media clips received from a plurality of client devices that include a first client device from which a first media clip with a first audio waveform is received and a second client device from which a second media clip with a second audio waveform is received;
determining a temporal overlap of the first media clip received from the first client device with the second media clip received from the second client device, the determining being performed by the one or more processors, the temporal overlap of the first and second media clips being determined by matching a first fingerprint of at least part of the first media clip with a second fingerprint of at least part of the second media clip;
merging the first media clip and the second media clip into a group of overlapping media clips based on the temporal overlap determined by the matching of the first fingerprint with the second fingerprint; and
generating the presentation to include the group formed by the merging of the first and second media clips based on the temporal overlap.
35. The non-transitory machine-readable medium of claim 34 , wherein:
each of the plurality of media clips is generated from an event by the plurality of client devices; and
the determining of the temporal overlap of the first media clip and the second media clip includes identifying the first media clip and the second media clip as being generated from the event.
36. The non-transitory machine-readable medium of claim 34 , wherein:
the first media clip and the second media clip are taken from the same event; and the operations further comprise
computing the first fingerprint from the first media clip taken from the same event and computing the second fingerprint from the second media clip taken from the same event.
37. The non-transitory machine-readable medium of claim 34 , wherein:
the first media clip and the second media clip are separately recorded views of a single event; and
the merging of the first media clip and the second media clip is based on a first audio fingerprint of the first media clip and on a second audio fingerprint of the second media clip.
38. The non-transitory machine-readable medium of claim 34 , wherein:
the first media clip includes first footage filmed during a concert at a performance hall by the first client device located in a first part of the performance hall;
the second media clip includes second footage filmed during the concert at the performance hall by the second client device located in a second part of the performance hall.
39. The non-transitory machine-readable medium of claim 34 , wherein:
the first media clip includes first footage of a concert at a performance hall filmed by the first client device from a first angle within the performance hall during a first period of time during the concert;
the second media clip includes second footage of the concert at the performance hall filmed by the second client device from a second angle within the performance hall during a second period of time that overlaps the first period of time during the concert.