COMPUTER-BASED SYSTEMS, DEVICES, AND METHODS FOR GENERATING MUSICAL COMPOSITIONS THAT ARE SYNCHRONIZED TO VIDEO
Computer-based systems, devices, and methods for generating musical compositions that are purposefully synchronized with video are described. A video timeline is defined with various time-markers that demarcate specific events in the video. A music timeline is generated based on the video timeline. The music timeline preserves the various time-markers from the video timeline. A computer-based musical composition system generates a musical composition based on the music timeline. The musical composition includes various musical events that align, synchronize, or coincide with the time-markers such that when the video and musical composition are played together the musical events align, synchronize, or coincide with the demarcated events in the video.
1 . A computer-implemented method of synchronizing a musical composition with a video, the method comprising:
preparing a video that begins at a video start time and ends at a video stop time;
establishing a first time-marker corresponding to a first event in the video, wherein the first time-marker demarcates a time in between the video start time and the video stop time at which the first event occurs in the video; and
generating a musical composition, wherein generating the musical composition includes synchronizing a first musical event of the musical composition with the first time-marker.
2 . The method of claim 1 wherein generating the musical composition further includes synchronizing a start time of the musical composition with the video start time.
3 . The method of claim 1 wherein generating the musical composition further includes synchronizing a stop time of the musical composition with the video stop time.
4 . The method of claim 1 wherein generating the musical composition further includes:
generating a timeline for the musical composition, the timeline comprising:
a start time for the musical composition;
a stop time for the musical composition; and
the first time-marker in between the start time for the musical composition and the stop time for the musical composition;
and
generating the musical composition based on the timeline, wherein the musical composition begins at the start time for the musical composition, includes the first musical event at the first time-marker, and ends at the stop time for the musical composition.
5 . The method of claim 4 , further comprising:
generating a video timeline for the video, wherein the video timeline includes the video start time, the video stop time, and the first time-marker in between the video start time and the video stop time, and wherein generating the musical composition based on the timeline includes aligning the timeline for the musical composition with the video timeline to synchronize the first musical event with the first event in the video at the first time-marker.
6 . The method of claim 1 , further comprising:
establishing at least one additional time-marker, each respective additional time-marker corresponding to a respective additional event in the video, wherein each respective additional time-marker demarcates a respective time in between the video start time and the video stop time at which a respective additional event occurs in the video, and wherein generating the musical composition further includes synchronizing a respective additional musical event of the musical composition with each respective additional time-marker.
7 . The method of claim 6 wherein generating the musical composition further includes:
generating a timeline for the musical composition, the timeline comprising:
a start time for the musical composition that coincides with the video start time;
a stop time for the musical composition that coincides with the video stop time;
the first time-marker in between the start time for the musical composition and the stop time for the musical composition; and
each additional time-marker in between the start time for the musical composition and the stop time for the musical composition;
and
generating the musical composition based on the timeline, wherein the musical composition begins at the start time for the musical composition, includes the first musical event at the first time-marker, includes each respective additional musical event at each respective additional time-marker, and ends at the stop time for the musical composition.
8 . The method of claim 1 wherein establishing a first time-marker corresponding to a first event in the video includes establishing the first time-marker corresponding to a first event selected from a group consisting of: a change of a view in the video, a change of a scene in the video, a change of a background in the video, a change of a foreground in the video, an action performed by a character in the video, an action performed by an object in the video, a period preceding a particular action in the video, a period preceding a particular event in the video, a period succeeding a particular action in the video, a period succeeding a particular event in the video, a change in a character depicted in the video, an introduction of a new character depicted in the video, a change in an object depicted in the video, and an introduction of a new object depicted in the video.
9 . The method of claim 1 wherein synchronizing a first musical event of the musical composition with the first time-marker includes synchronizing, with the first time-marker, a musical event selected from a group consisting of: a particular note, a particular note sequence, a key modulation, a chord progression, a particular instrument, a change in instrumentation, a change in beats per minute, a change in timing, an acceleration, a deceleration, and a change in volume.
10 . The method of claim 1 wherein generating a musical composition includes automatically generating the musical composition by a computer-based musical composition system, and wherein automatically generating the musical composition by the computer-based musical composition system includes defining a start time of the musical composition to at least approximately coincide with the video start time, defining a stop time of the musical composition to at least approximately coincide with the video stop time, defining the first musical event of the musical composition to coincide with the first time-marker, and generating music that includes the first musical event at the first time-marker and continuously connects between the start time of the musical composition and the stop time of the musical composition.
11 . The method of claim 1 wherein generating a musical composition includes solving, by a computer-based musical composition system, a constraint satisfaction problem, and wherein synchronizing a first musical event of the musical composition with the first time-marker includes providing, in a formulation of the constraint satisfaction problem, a constraint that specifies that the first musical event of the musical composition is synchronized with the first time-marker.
12 . A system for synchronizing a musical composition with a video, the system comprising:
at least one processor; and
a non-transitory processor-readable storage medium communicatively coupled to the at least one processor, the non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to:
prepare a video that begins at a video start time and ends at a video stop time;
establish a first time-marker that demarcates a time in between the video start time and the video stop time at which a first event occurs in the video; and
generate a musical composition that synchronizes a first musical event with the first time-marker.
13 . The system of claim 12 wherein the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a musical composition that synchronizes a first musical event with the first time-marker, cause the at least one processor to:
generate a timeline for the musical composition, the timeline comprising:
a start time for the musical composition;
a stop time for the musical composition; and
the first time-marker in between the start time for the musical composition and the stop time for the musical composition;
and
generate the musical composition based on the timeline, wherein the musical composition begins at the start time for the musical composition, includes the first musical event at the first time-marker, and ends at the stop time for the musical composition.
14 . The system of claim 13 wherein the non-transitory processor-readable storage medium further stores processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a video timeline for the video, wherein the video timeline includes the video start time, the video stop time, and the first time-marker in between the video start time and the video stop time, and wherein the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate the musical composition based on the timeline, cause the at least one processor to align the timeline for the musical composition with the video timeline to synchronize the first musical event with the first event in the video at the first time-marker.
15 . The system of claim 12 wherein:
the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to establish a first time-marker that demarcates a time in between the video start time and the video stop time at which a first event occurs in the video, further cause the at least one processor to establish a second time-marker that demarcates a time in between the video start time and the video stop time at which a second event occurs in the video; and
the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a musical composition that synchronizes a first musical event synchronized with the first time-marker, further cause the at least one processor to generate the musical composition with a second musical event synchronized with the second time-marker.
16 . The system of claim 15 wherein the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a musical composition that synchronizes a first musical event with the first time-marker, cause the at least one processor to:
define a constraint satisfaction problem, the constraint satisfaction problem including:
a first constraint that specifies that the first musical event of the musical composition is synchronized with the first time-marker;
and
a second constraint that specifies that the second musical event of the musical composition is synchronized with the second time-marker;
and
solve the constraint satisfaction problem.
17 . A computer program product comprising:
processor-executable instructions and/or data that, when the computer program product is stored in a non-transitory processor-readable storage medium and executed by at least one processor communicatively coupled to the non-transitory processor-readable storage medium, cause the at least one processor to:
prepare a video that begins at a video start time and ends at a video stop time;
establish a first time-marker that demarcates a time in between the video start time and the video stop time at which a first event occurs in the video; and
generate a musical composition that synchronizes a first musical event with the first time-marker.
18 . The computer program product of claim 17 wherein the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a musical composition that synchronizes a first musical event with the first time-marker, cause the at least one processor to:
generate a timeline for the musical composition, the timeline comprising:
a start time for the musical composition;
a stop time for the musical composition; and
the first time-marker in between the start time for the musical composition and the stop time for the musical composition;
and
generate the musical composition based on the timeline, wherein the musical composition begins at the start time for the musical composition, includes the first musical event at the first time-marker, and ends at the stop time for the musical composition.
19 . The computer program product of claim 18 wherein the non-transitory processor-readable storage medium further stores processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a video timeline for the video, wherein the video timeline includes the video start time, the video stop time, and the first time-marker in between the video start time and the video stop time, and wherein the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate the musical composition based on the timeline, cause the at least one processor to align the timeline for the musical composition with the video timeline to synchronize the first musical event with the first event in the video at the first time-marker.
20 . The computer program product of claim 17 wherein:
the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to establish a first time-marker that demarcates a time in between the video start time and the video stop time at which a first event occurs in the video, further cause the at least one processor to establish a second time-marker that demarcates a time in between the video start time and the video stop time at which a second event occurs in the video; and
the processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to generate a musical composition that synchronizes a first musical event synchronized with the first time-marker, further cause the at least one processor to generate the musical composition with a second musical event synchronized with the second time-marker.