IP Library Granted Patent US 11,601,698
Granted Patent B2
US 11,601,698 · App. 17/217,863 · Granted Mar 7, 2023

Intelligent synchronization of media streams

Inventors: Brian Gramo (Santa Monica, CA); Arthur Kmbikyan (Santa Monica, CA); David Steigelfest (Santa Monica, CA)
Assignee: SUPER LEAGUE GAMING, INC.
H04N21/242A63F13/35H04N21/233H04N21/23406
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,601,698
App. No.
17/217,863
Granted
Mar 7, 2023
Kind
B2
Abstract

Systems and methods for intelligent synchronization of media streams are provided. A server may receive streams corresponding to an interactive session and sent over a communication network from user devices in the interactive session. A predetermined attribute may be identified as present in each of the streams, but received at different times by the server. The server may determine a time difference between a time that a predetermined attribute in a first stream of the streams was received and a time that the predetermined attribute in a second stream of the streams was received. The first stream and the second stream may then be synchronized using the time difference and provided to a recipient device.

Claims (47)

1. A method for stream synchronization, the method comprising:

receiving a plurality of streams at a server, the streams corresponding to an interactive session and sent over a communication network from a plurality of user devices in the interactive session;

identifying a predetermined attribute present in each of the streams, wherein the predetermined attribute within each stream is received by the server at different times;

determining a time difference between a time that a predetermined attribute in a first one of the streams was received by the server and a time that the predetermined attribute in a second one of the streams was received by the server;

synchronizing the first stream and the second stream using the time difference, wherein the predetermined attribute within the first stream is synchronized to the predetermined attribute in the second stream; and

providing the synchronized first and second stream over the communication network to a recipient device.

2. The method of claim 1 , wherein each of the streams includes at least one of an audio signal and a video signal, and wherein the predetermined attribute includes at least one of a visual cue or an audio cue.

3. The method of claim 1 , wherein the predetermined attribute includes an in-game cue corresponding to an occurrence of one or more in-game events during the interactive session, and wherein identifying the predetermined attribute includes detecting the occurrence based on game data in the respective stream.

4. The method of claim 1 , wherein identifying the predetermined attribute includes comparing an audio waveform of the respective stream to a predetermined audio waveform.

5. The method of claim 4 , wherein the predetermined audio waveform has a wave property that changes in accordance with a time offset from a reference time, wherein determining the time difference is based on a time offset for the first stream and a time offset for the second stream.

6. The method of claim 5 , further comprising introducing an audio signal corresponding to the predetermined audio waveform into the interactive session.

7. The method of claim 1 , wherein the predetermined attribute corresponds one or more in-stream events, and wherein identifying the predetermined attribute includes detecting the in-stream events based on metadata in the respective stream.

8. The method of claim 1 , wherein synchronizing the first stream and the second stream using the time difference includes providing a buffer for at least one of the first stream and the second stream, wherein a size of the buffer corresponds to the time difference.

9. The method of claim 1 , wherein synchronizing the first stream and the second stream using the time difference comprises:

writing a portion of the first stream or the second stream to a memory, wherein the portion corresponds to the time difference; and

reading the portion from the memory after a time period corresponding to the time difference.

10. The method of claim 1 , further comprising:

applying artificial intelligence to detect a common pattern in the streams;

identifying at least a part of the pattern as a new predetermined attribute; and

storing the new predetermined attribute in memory, wherein the new predetermined attribute is used to synchronize streams in a subsequent interactive session.

11. A system for stream synchronization comprising:

a communication interface that receives a plurality of streams, the streams corresponding to an interactive session and sent over a communication network from a plurality of user devices in the interactive session;

a processor that executes instructions stored in memory, wherein the processor executes the instructions to:

identify a predetermined attribute present in each of the streams, wherein the predetermined attribute within each stream is received at different times;

determine a time difference between a time that a predetermined attribute in a first one of the streams was received and a time that the predetermined attribute in a second one of the streams was received by the server; and

synchronize the first stream and the second stream using the time difference, wherein the predetermined attribute within the first stream is synchronized to the predetermined attribute in the second stream, and

wherein the communication interface provides the synchronized first and second stream over the communication network to a recipient device.

12. The system of claim 11 , wherein each of the streams includes at least one of an audio signal and a video signal, and wherein the predetermined attribute includes at least one of a visual cue or an audio cue.

13. The system of claim 11 , wherein the predetermined attribute includes an in-game cue corresponding to an occurrence of one or more in-game events during the interactive session, and wherein the processor identifies the predetermined attribute by detecting the occurrence based on game data in the respective stream.

14. The system of claim 11 , wherein the processor identifies the predetermined attribute by comparing an audio waveform of the respective stream to a predetermined audio waveform.

15. The system of claim 14 , wherein the predetermined audio waveform has a wave property that changes in accordance with a time offset from a reference time, wherein the processor determines the time difference based on a time offset for the first stream and a time offset for the second stream.

16. The system of claim 15 , wherein the processor executes further instructions to introduce an audio signal corresponding to the predetermined audio waveform into the interactive session.

17. The system of claim 11 , wherein the predetermined attribute corresponds one or more in-stream events, and wherein the processor identifies the predetermined attribute by detecting the in-stream events based on metadata in the respective stream.

18. The system of claim 11 , further comprising memory, wherein the processor synchronizes the first stream and the second stream using the time difference by providing a buffer in the memory for at least one of the first stream and the second stream, wherein a size of the buffer corresponds to the time difference.

19. The system of claim 11 , further comprising memory, wherein the processor synchronizes the first stream and the second stream using the time difference by:

writing a portion of the first stream or the second stream to the memory, wherein the portion corresponds to the time difference; and

reading the portion from the memory after a time period corresponding to the time difference.

20. The system of claim 11 , further comprising memory, wherein the processor executes further instructions to:

apply artificial intelligence to detect a common pattern in the streams; and

identify at least a part of the pattern as a new predetermined attribute;

wherein the memory stores the new predetermined attribute, and wherein the new predetermined attribute is used to synchronize streams in a subsequent interactive session.

21. A non-transitory machine-readable storage medium having instructions embodied thereon, the instructions executable by a processor to perform a method, the method comprising:

receiving a plurality of streams at a server, the streams corresponding to an interactive session and sent over a communication network from a plurality of user devices in the interactive session;

identifying a predetermined attribute present in each of the streams, wherein the predetermined attribute within each stream is received by the server at different times;

determining a time difference between a time that a predetermined attribute in a first one of the streams was received by the server and a time that the predetermined attribute in a second one of the streams was received by the server;

synchronizing the first stream and the second stream using the time difference, wherein the predetermined attribute within the first stream is synchronized to the predetermined attribute in the second stream; and

providing the synchronized first and second stream over the communication network to a recipient device.

Assignments (2)
SECURITY INTEREST Recorded Jan 18, 2024
From: SUPER LEAGUE GAMING, INC.
To: SLR DIGITAL FINANCE LLC
Reel/Frame 066171/0432 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2021
From: GRAMO, BRIAN; KMBIKYAN, ARTHUR; STEIGELFEST, DAVID
To: SUPER LEAGUE GAMING, INC.
Reel/Frame 056568/0809 →
Continuity (1)
Related Publication 20220321933A1 · Oct 6, 2022
Cited By (2)
US 12,452,477 US 12,500,997