IP Library Granted Patent US 10,522,190
Granted Patent B2
US 10,522,190 · App. 15/975,103 · Granted Dec 31, 2019

Methods and apparatus for ordered serial synchronization of multimedia streams upon sensor changes

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,522,190
App. No.
15/975,103
Granted
Dec 31, 2019
Kind
B2
Abstract

An apparatus includes a processor with first and second input ports and a memory operably coupled to the processor. The processor can detect streams of media samples at the input ports and determine, in response to the detection of the streams of media samples, a capture start time. The processor can also capture a first frame of a first stream of media samples beginning at the capture start time, and a first frame of a second stream of media samples beginning at a first time subsequent to the capture start time. The processor can also calculate a relative offset time based on the capture start time, the first time, and a rate associated with the second stream of media samples, and store, in the memory, an indication of an association between the captured first frame of the second stream of media samples and the relative offset time.

Claims (68)

1. An apparatus, comprising:

a processor having a first input port and a second input port; and

a memory operably coupled to the processor,

the processor configured to:

detect, at the first input port, a first stream of media samples;

detect, at the second input port, a second stream of media samples;

determine, in response to the detection of the first stream of media samples and the second stream of media samples, a capture start time associated with the first stream of media samples;

capture, beginning at the capture start time, a first frame of the first stream of media samples;

capture, beginning at a first time subsequent to the capture start time, a first frame of the second stream of media samples;

calculate a relative offset time based on the capture start time, the first time, and a rate associated with the second stream of media samples;

store, in the memory, an indication of an association between the captured first frame of the second stream of media samples and the relative offset time;

combine the captured first frame of the first stream of media samples and the captured first frame of the second stream of media samples, based on the relative offset time, to form a synchronized media file; and

transmit the synchronized media file to a remote compute device.

2. The apparatus of claim 1 , wherein the processor is further configured to receive an indication of the capture start time from a control server in operable communication with the processor.

3. The apparatus of claim 1 , wherein the processor is further configured to receive an indication of the capture start time from a local scheduler.

4. The apparatus of claim 1 , wherein the relative offset time is a first relative offset time, the processor further configured to:

detect, at a third input port of the processor, a third stream of media samples;

capture, beginning at a second time subsequent to the first time, a first frame of the third stream of media samples;

calculate a second relative offset time based on the capture start time, the second time, and a rate associated with the third stream of media samples; and

store, in the memory, an indication of an association between the captured first frame of the third stream of media samples and the second relative offset time.

5. The apparatus of claim 4 , wherein the processor is further configured to form the synchronized media file by combining the captured first frame of the third stream of media samples with the captured first frame of the first stream of media samples and the captured first frame of the second stream of media samples.

6. The apparatus of claim 1 , wherein the processor is further configured to:

capture a plurality of additional frames of the first stream of media samples;

capture a plurality of additional frames of the second stream of media samples; and

form the synchronized media file by combining the plurality of additional frames of the first stream of media samples and the plurality of additional frames of the second stream of media samples with the captured first frame of the first stream of media samples and the captured first frame of the second stream of media samples, based on the relative offset time.

7. The apparatus of claim 1 , wherein the processor is further configured to identify the first time based on a hardware timestamp encoded by the second stream of media samples.

8. A method, comprising:

detecting, at a first input port of a processor, a rate of a first stream of media samples;

detecting, at a second input port of the processor, a rate of a second stream of media samples;

detecting, at a third input port of the processor, a rate of a third stream of media samples;

initiating a capture period having a capture start time based on the first stream of media samples;

during the capture period:

capturing, beginning at the capture start time, a first frame of the first stream of media samples, based on the rate of the first stream of media samples being higher than the rate of the second stream of media samples and the rate of the third stream of media samples;

capturing, beginning at a first time subsequent to the capture start time, a first frame of the second stream of media samples, based on the rate of the second stream of media samples being higher than the rate of the third stream of media samples;

capturing, beginning at a second time subsequent to the first time, a first frame of the third stream of media samples;

calculating a relative offset time of the captured first frame of the second stream of media samples; and

calculating a relative offset time of the captured first frame of the third stream of media samples,

and

combining the captured first frame of the first stream of media samples, the captured first frame of the second stream of media samples, and the captured first frame of the third stream of media samples, based on the relative offset time of the captured first frame of the second stream of media samples and the relative offset time of the captured first frame of the third stream of media samples, to form a synchronized, streamable media file.

9. The method of claim 8 , wherein the first stream of media samples includes a plurality of sequenced audio samples, the second stream of media samples includes a plurality of sequenced video frames, and the third stream of media samples includes a plurality of sequenced desktop frames.

10. The method of claim 8 , wherein the synchronized, streamable media file is a first synchronized, streamable media file, the method further comprising:

detecting a fourth stream of media samples received at the processor via a fourth input port;

capturing the fourth stream of media samples; and

generating a second synchronized, streamable media file that includes the fourth stream of media samples.

11. The method of claim 8 , wherein the initiating the capture period is in response to a communication received from a local scheduler.

12. The method of claim 8 , wherein the initiating the capture period is in response to a communication received from a control server.

13. The method of claim 8 , further comprising storing, in the memory:

an indication of an association between the captured first frame of the second stream of media samples and the relative offset time of the captured first frame of the second stream of media samples; and

an indication of an association between the captured first frame of the third stream of media samples and the relative offset time of the captured first frame of the third stream of media samples.

14. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:

combine a first captured portion of a first stream of media samples, a first captured portion of a second stream of media samples, and a first captured portion of a third stream of media samples, based on (1) a first relative offset time associated with the second stream of media samples, and (2) a second relative offset time associated with the third stream of media samples, to form a first synchronized, streamable media file;

detect an interruption of the second stream of media samples;

capture, in response to detecting the interruption, a second captured portion of the first stream of media samples, and a second captured portion of the third stream of media samples;

calculate, in response to detecting the interruption, a revised relative offset time for the third stream of media samples;

generate a second synchronized, streamable media file based on the revised relative offset time;

detect a fourth stream of media samples received at the processor;

capture the fourth stream of media samples; and

generate a third synchronized, streamable media file that includes the fourth stream of media samples.

15. The non-transitory processor-readable medium of claim 14 , wherein the code further comprises code to cause the processor to:

receive an instruction to initiate a capture period;

capture, in response to the instruction and beginning at a capture start time, the first captured portion of the first stream of media samples via a first input port of the processor;

capture, beginning at a first time subsequent to the capture start time, the first captured portion of the second stream of media samples via a second input port of the processor;

capture, beginning at a second time subsequent to the first time, the first captured portion of the third stream of media samples via a third input port of the processor;

calculate, via the processor, the first relative offset time and the second relative offset time; and

store, in a memory, a representation of the first relative offset time and a representation of the second relative offset time.

16. The non-transitory processor-readable medium of claim 15 , wherein the instruction to initiate a capture period is received from one of a local scheduler or a control server.

17. The non-transitory processor-readable medium of claim 14 , wherein a frame rate of the first stream of media samples is higher than a frame rate of the second stream of media samples, and the frame rate of the second stream of media samples is higher than a frame rate of the third stream of media samples.

18. The non-transitory processor-readable medium of claim 14 , wherein the code further comprises code to cause the processor to transmit the first synchronized media file to a remote compute device.

Assignments (5)
SECURITY INTEREST Recorded May 12, 2025
From: TURNING TECH INTERMEDIATE, INC.; TURNING ACQUISITION, INC.; ECHO 360, INC.; ECHO 360 HOLDINGS, INC.; ECHO 360 CONTINUING EDUCATION, LLC; TURNING TECH LLC; TURNING TECH HOLDINGS, LLC; TURNING TECHNOLOGIES, LLC; RESPONSE TOOLS, LLC; INKLING SYSTEMS, INC.; SPEAKWORKS, INC.; RESPONSIVE INNOVATIONS, LLC
To: ADVANTAGE CAPITAL MANAGEMENT LLC
Reel/Frame 071276/0781 →
RELEASE OF SECURITY INTEREST Recorded May 1, 2024
From: RUNWAY GROWTH CREDIT FUND INC.
To: ECHO 360, INC.
Reel/Frame 067286/0797 →
SECURITY INTEREST Recorded May 1, 2024
From: ECHO 360, INC.; ECHO 360 CONTINUING EDUCATION, LLC; ECHO 360 HOLDINGS, INC.; TURNING ACQUISITION, INC.; TURNING TECH INTERMEDIATE, INC.; TURNING TECH LLC; TURNING TECH HOLDINGS, LLC; TURNING TECHNOLOGIES, LLC; RESPONSIVE INNOVATIONS, LLC; RESPONSE TOOLS, LLC; INKLING SYSTEMS, INC.
To: ADVANTAGE CAPITAL MANAGEMENT LLC, AS ADMINSTRATIVE AGENT
Reel/Frame 067289/0001 →
SECURITY INTEREST Recorded May 7, 2019
From: ECHO 360, INC.
To: RUNWAY GROWTH CREDIT FUND INC.
Reel/Frame 049103/0766 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2018
From: REYNOLDS, MARK; PROCTER, SHAUN; CRAMER, ANDREW; CHUDIK, MARTIN
To: ECHO360, INC.
Reel/Frame 046693/0409 →