IP Library Granted Patent US 10,757,468
Granted Patent B2
US 10,757,468 · App. 16/222,481 · Granted Aug 25, 2020

Systems and methods for performing playout of multiple media recordings based on a matching segment among the recordings

Inventors: Avery Wang (Palo Alto, CA); Maxwell Leslie Szabo (San Francisco, CA)
Assignee: Apple Inc.
H04N21/4307G06F16/41G06F16/71G06F16/7834G11B27/007G11B27/10H04N9/802H04N21/4394H04N21/44008H04N21/8456
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,757,468
App. No.
16/222,481
Granted
Aug 25, 2020
Kind
B2
Abstract

An example method for performing playout of multiple media recordings includes receiving a plurality of media recordings, indexing the plurality of media recordings for storage into a database, dividing each of the plurality of media recordings into multiple segments, and for each segment of each media recording, (i) comparing the segment with the indexed plurality of media recordings stored in the database to determine one or more matches to the segment, and (ii) determining a relative time offset of the segment within each matched media recording. Following, the method includes performing playout of a representation of the plurality of media recordings based on the relative time offset of each matched segment.

Claims (65)

1. A method comprising:

receiving a plurality of media recordings, wherein each of the plurality of media recordings is associated with a master timeline;

indexing the plurality of media recordings for storage into a database;

dividing each of the plurality of media recordings into multiple segments;

for each segment of each media recording,

comparing the segment with the indexed plurality of media recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched media recording;

mapping the relative time offset of segments within a given media recording to the master timeline; and

for each of the plurality of media recordings, determining a timescale ratio that is a speed adjustment ratio that indicates an amount that the respective media recording needs to be sped up or slowed down to map to the master timeline.

2. The method of claim 1 , wherein receiving the plurality of media recordings comprises receiving a plurality of video recordings;

wherein indexing the plurality of media recordings for storage into the database comprises indexing the plurality of video recordings; and

for each segment of each video recording,

comparing audio of the segment with the indexed plurality of video recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched video recording; and

performing playout of the representation of the plurality of video recordings based on the relative time offset of each matched segment.

3. The method of claim 1 , wherein the plurality of media recordings are videos, and wherein receiving the plurality of media recordings comprises receiving the plurality of media recordings from multiple video cameras.

4. The method of claim 1 , further comprising: performing playout of a representation of the plurality of media recordings based on the relative time offset of each matched segment.

5. The method of claim 1 , further comprising: stretching one or more of the plurality of media recordings to map to the master timeline based on one or more of the plurality of media recordings being captured at different rates.

6. The method of claim 1 , wherein the plurality of media recordings include video, and the method further comprises:

stretching one or more of the plurality of media recordings to play back in synchrony with each other to map to the master timeline.

7. The method of claim 1 , further comprising: interpolating one or more of the plurality of media recordings to play back in synchrony with each other to map to the master timeline.

8. A non-transitory computer-readable storage media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform functions comprising:

receiving a plurality of media recordings, wherein each of the plurality of media recordings is associated with a master timeline;

indexing the plurality of media recordings for storage into a database;

dividing each of the plurality of media recordings into multiple segments;

for each segment of each media recording,

comparing the segment with the indexed plurality of media recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched media recording;

mapping the relative time offset of segments within a given media recording to the master timeline; and

for each of the plurality of media recordings, determining a timescale ratio that is a speed adjustment ratio that indicates an amount that the respective media recording needs to be sped up or slowed down to map to the master timeline.

9. The non-transitory computer-readable storage media of claim 8 , wherein the function of receiving the plurality of media recordings comprises receiving a plurality of video recordings;

wherein the function of indexing the plurality of media recordings for storage into the database comprises indexing the plurality of video recordings; and

wherein the instructions further cause the one or more processors to perform functions comprising:

for each segment of each video recording,

comparing audio of the segment with the indexed plurality of video recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched video recording; and

performing playout of the representation of the plurality of video recordings based on the relative time offset of each matched segment.

10. The non-transitory computer-readable storage media of claim 8 , wherein the plurality of media recordings are videos, and wherein receiving the plurality of media recordings comprises receiving the plurality of media recordings from multiple video cameras.

11. The non-transitory computer-readable storage media of claim 8 , wherein the instructions further cause the one or more processors to perform functions comprising: performing playout of a representation of the plurality of media recordings based on the relative time offset of each matched segment.

12. The non-transitory computer-readable storage media of claim 8 , wherein the instructions further cause the one or more processors to perform functions comprising: stretching one or more of the plurality of media recordings to map to the master timeline.

13. The non-transitory computer-readable storage media of claim 8 , wherein the plurality of media recordings include video, and wherein the instructions further cause the one or more processors to perform functions comprising:

stretching one or more of the plurality of media recordings to play back in synchrony with each other to map to the master timeline based on one or more of the plurality of media recordings being captured at different rates, wherein stretching includes repeating playout of frames.

14. The non-transitory computer-readable storage media of claim 8 , wherein the instructions further cause the one or more processors to perform functions comprising: interpolating one or more of the plurality of media recordings to play back in synchrony with each other to map to the master timeline.

15. A computing device comprising:

one or more processors; and

non-transitory computer-readable media storing instructions that, when executed, by the one or more processors, cause the one or more processors to perform functions comprising:

receiving a plurality of media recordings, wherein each of the plurality of media recordings is associated with a master timeline;

indexing the plurality of media recordings for storage into a database;

dividing each of the plurality of media recordings into multiple segments;

for each segment of each media recording,

comparing the segment with the indexed plurality of media recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched media recording;

mapping the relative time offset of segments within a given media recording to the master timeline; and

for each of the plurality of media recordings, determining a timescale ratio that is a speed adjustment ratio that indicates an amount that the respective media recording needs to be sped up or slowed down to map to the master timeline.

16. The computing device of claim 15 , wherein the function of receiving the plurality of media recordings comprises receiving a plurality of video recordings;

wherein the function of indexing the plurality of media recordings for storage into the database comprises indexing the plurality of video recordings; and

wherein the instructions further cause the one or more processors to perform functions comprising:

for each segment of each video recording,

comparing audio of the segment with the indexed plurality of video recordings stored in the database to determine one or more matches to the segment; and

determining a relative time offset of the segment within each matched video recording; and

performing playout of the representation of the plurality of video recordings based on the relative time offset of each matched segment.

17. The computing device of claim 15 , wherein the plurality of media recordings are videos, and wherein receiving the plurality of media recordings comprises receiving the plurality of media recordings from multiple video cameras.

18. The computing device of claim 15 , wherein the instructions further cause the one or more processors to perform functions comprising: performing playout of a representation of the plurality of media recordings based on the relative time offset of each matched segment.

19. The computing device of claim 15 , wherein the instructions further cause the one or more processors to perform functions comprising: stretching one or more of the plurality of media recordings to map to the master timeline.

20. The computing device of claim 15 , wherein the instructions further cause the one or more processors to perform functions comprising: interpolating one or more of the plurality of media recordings to play back in synchrony with each other to map to the master timeline.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 3, 2020
From: SHAZAM INVESTMENTS LIMITED
To: APPLE INC.
Reel/Frame 053689/0704 →
Continuity (2)
Continuation 15729104 · Oct 10, 2017
Related Publication 20190141388A1 · May 9, 2019