IP Library › Granted Patent US 8,347,210
Granted Patent B2
US 8,347,210 · App. 12/563,704 · Granted Jan 1, 2013

Synchronizing video with audio beats

Assignee: Apple Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,347,210
App. No.
12/563,704
Granted
Jan 1, 2013
Kind
B2
Abstract

A computer-implemented method to synchronize video with beats in audio is described. Multiple video frames and multiple audio frames are displayed in a first and second bounded region, respectively, in a user interface. The multiple audio frames represent audio content that includes a periodic beat that is audible when the audio content is played. Input is received to associate one or more video frames with the multiple audio frames using a cursor. In response to the input, the first video frame is synchronized with a beat audio frame in the multiple audio frames. In response to a playback input, the first video frame is displayed in the user interface when the beat audio frame is played.

Claims (32)

1. A computer-implemented method comprising:

displaying a first bounded region representing a plurality of video frames in a first area of a user interface;

displaying a second bounded region representing a plurality of audio frames in a second area of the user interface, wherein the plurality of audio frames represent audio content that includes two beat audio frames corresponding to two consecutive beats in the audio content, and wherein a left edge and a right edge of the second bounded region represent the two consecutive beats;

receiving input to associate one or more video frames of the plurality of video frames with the plurality of audio frames, the input comprising a positioning of a cursor over a portion of the first bounded region and over a portion of the second bounded region; and

in response to the received input, synchronizing a first video frame of the one or more video frames with a first beat audio frame associated with the second bounded region.

2. The method of claim 1 , wherein the first beat audio frame is included in the plurality of audio frames such that in response to a playback input, the first video frame is displayed in the user interface when the first beat audio frame is played.

3. The method of claim 1 , further comprising:

providing the plurality of audio frames to a beat detection system configured to identify beat audio frames; and

receiving the two beat audio frames from the beat detection system.

4. The method of claim 1 , wherein a video frame of the plurality of video frames is associated with a first position of a plurality of first positions that collectively represent a horizontal dimension of the first bounded region, and wherein an audio frame of the plurality of audio frames is associated with a second position of a plurality of second positions that collectively represent a horizontal dimension of the second bounded region.

5. The method of claim 4 , wherein positioning the cursor over a first portion of the first bounded region corresponds to a selection of a video frame associated with that position, and positioning the cursor over a second portion of the second bounded region corresponds to a selection of an audio frame associated with the second position.

6. The computer-readable medium of claim 5 , the operations further comprising:

receiving playback input to play the video content to which the audio content is synchronized; and

in response to receiving the playback input, playing back the audio content and the video content, the playing back comprising displaying the video content within the bounded region while simultaneously playing the synchronized audio content.

7. The computer-readable medium of claim 6 , the operations further comprising:

while playing back the audio content and the video content, displaying a vertical line spanning a vertical dimension of the bounded region over the video content; and

traversing the vertical line across a horizontal dimension of the bounded region such that when the audio content represented by the bounded region ends, the vertical line reaches a right edge of the bounded region.

8. The computer-readable medium of claim 5 , the operations further comprising:

providing the plurality of audio frames to a beat detection system configured to identify the two beat audio frames; and

receiving the two beat audio frames from the beat detection system.

9. A computer-readable medium, tangibly embodying a computer program product which, when executed by one or more computers performs operations comprising:

displaying a bounded region representing audio content including a plurality of audio frames including two beat audio frames that represent two consecutive beats in the audio content, the bounded region associated with a beat duration between the two beat audio frames, wherein a left edge and a right edge of the bounded region represent the two consecutive beats;

receiving a plurality of video frames representing video content to associate with the plurality of audio frames, wherein a video duration between a first video frame and a last video frame of the plurality of video frames represents a duration of the video content;

associating a first video frame in the video content to a first beat audio frame of the two beat audio frames to synchronize the video content and the audio content;

determining that a duration of the video content is less than or equal to the beat duration; and

based on the determining, displaying a video frame of the plurality of video frames within the bounded region.

10. A computer-implemented method comprising:

determining beat audio frames included in an audio content, a beat audio frame being an audio frame that includes a beat, wherein a beat is a pulse included in the audio content;

displaying a beat region in a first area of a user interface, the beat region being a bounded region of the user interface providing a visual representation of a fraction of audio frames in the audio content, wherein a left edge and a right edge of the beat region represent two consecutive beats in the audio content and a horizontal distance between the left and right edges represents a time duration between two beat audio frames corresponding to the two consecutive beats;

displaying a thumbnail in a second area of the user interface, the thumbnail being a visual representation of a fraction of video frames included in a video content;

receiving an user input for associating one or more video frames represented by a portion of the thumbnail with one or more audio frames represented by a portion of the beat region, the input comprising a selection of the portion of the thumbnail and a selection of the portion of the beat region; and

responsive to receiving the user input, synchronizing the one or more video frames represented by the selected portion of the thumbnail with the one or more audio frames represented by the selected portion of beat region such that during a playback of the one or more video frames represented by the selected portion of the thumbnail, the one or more audio frames represented by the selected portion of beat region are also played back simultaneously.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2009
From: UBILLOS, RANDY; PERRODIN, LAURENT
To: APPLE INC.
Reel/Frame 023357/0795 →
Continuity (2)
Provisional Application 61100631 · Sep 26, 2008
Related Publication 20100080532A1 · Apr 1, 2010