Rolling buffer for retroactive capture of clips
The disclosed technology addresses the need in the art for to facilitate user accounts participating in live audio-video sessions such as voice and/or video calls, or watching game play remotely to capture a clip of the video call or game play after seeing some event during the live audio-video session. The present technology can maintain a rolling buffer of the audio-video streams such that a user account can retroactively capture a clip of the audio-video streams after it has already been presented.
1 . A method comprising:
receiving audio-video streams in a live audio-video session, the audio-video streams including video frames and at least two streams of audio associated with the video frames;
storing the video frames and the at least two streams of audio in a rolling buffer that overwrites older data as new data is received;
receiving an instruction to save a clip of the audio-video streams, the clip to consist of at least a portion of the audio-video streams that has already occurred and is stored in the rolling buffer;
while continuing to receive audio-video streams in the live audio-video session,
retroactively saving the at least the portion of the audio-video streams stored in the rolling buffer in a persistent memory for editing, whereby the at least the portion of the audio-video streams is saved after it has already been presented; and
providing an graphical user interface to edit the video frames and the at least two streams of audio from the portion of the audio-video streams stored in the persistent memory, wherein the graphical user interface to edit provides an edited clip.
2 . The method of claim 1 , further comprising:
receiving a selection of a first audio stream of the at least two streams of audio for editing, wherein respective audio streams within the at least two streams of audio can be independently edited.
3 . The method of claim 2 , during the editing, superimposing user interface attributes associated with a client application over the edited video.
4 . The method of claim 1 , wherein the storing the video frames in the rolling buffer further comprises:
encoding the video frames using a first encoder, wherein the first encoder dynamically selects a keyframe interval.
5 . The method of claim 4 , further comprising:
deleting a portion of the video frames from the rolling buffer by identifying an oldest keyframe in the rolling buffer and a second older keyframe in the rolling buffer, and deleting the oldest keyframe and the video frames between the oldest keyframe and the second oldest keyframe.
6 . The method of claim 1 , further comprising:
receiving an instruction to initiate a clips capture tool prior to the storing the video frames and the at least two streams of audio in the rolling buffer.
7 . The method of claim 1 , further comprising:
determining that the video frames and the at least two streams of audio in the rolling buffer are likely to be selected for use in the clip;
dynamically increasing a size of the rolling buffer to avoid deletion of portions of the rolling buffer.
8 . The method of claim 1 , further comprising managing a size of the rolling buffer by adjusting a frame rate of captured video frames, or when there are multiple video streams in the audio-video streams, storing video frames from an in focus video stream, or a selected video stream, of the multiple video streams, but not other video streams of the multiple video streams.
9 . The method of claim 1 , further comprising:
encoding the video frames, where the encoding the video frames includes:
determining an encoding pipeline that is optimized to a client device in which a client application is executing, the determining comprising:
identifying whether a hardware encoder is available;
selecting the hardware encoder when the hardware encoder is available.
10 . A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
receive audio-video streams in a live audio-video session, the audio-video streams including video frames and at least two streams of audio associated with the video frames;
store the video frames and the at least two streams of audio in a rolling buffer;
receive an instruction to save a clip of the audio-video streams, the clip to consist of at least a portion of the audio-video streams that has already occurred and is stored in the rolling buffer;
while continuing to receive audio-video streams in the live audio-video session,
retroactively save the at least the portion of the audio-video streams stored in the rolling buffer in a persistent memory for editing, whereby the at least the portion of the audio-video streams is saved after it has already been presented; and
provide ana graphical user interface to edit the video frames and the at least two streams of audio from the portion of the audio-video streams stored in the persistent memory, wherein the graphical user interface to edit provides an edited clip.
11 . The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:
receive a selection of a first audio stream of the at least two streams of audio for editing, wherein respective audio streams within the at least two streams of audio can be independently edited.
12 . The computing apparatus of claim 10 , wherein the storing the video frames in the rolling buffer further comprises:
encode the video frames using a first encoder, wherein the first encoder dynamically selects a keyframe interval.
13 . The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:
determine that the video frames and the at least two streams of audio in the rolling buffer are likely to be selected for use in the clip;
dynamically increase a size of the rolling buffer to avoid deletion of portions of the rolling buffer.
14 . The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to manage a size of the rolling buffer by adjusting a frame rate of captured video frames, or when there are multiple video streams in the audio-video streams, storing video frames from an in focus video stream, or a selected video stream, of the multiple video streams, but not other video streams of the multiple video streams.
15 . The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:
encode the video frames, where the encoding the video frames includes:
determine an encoding pipeline that is optimized to a client device in which a client application is executing, the determining comprising:
identify whether a hardware encoder is available;
select the hardware encoder when the hardware encoder is available.
16 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
receive audio-video streams in a live audio-video session, the audio-video streams including video frames and at least two streams of audio associated with the video frames;
store the video frames and the at least two streams of audio in a rolling buffer;
receive an instruction to save a clip of the audio-video streams, the clip to consist of at least a portion of the audio-video streams that has already occurred and is stored in the rolling buffer;
while continuing to receive audio-video streams in the live audio-video session,
retroactively save the at least the portion of the audio-video streams stored in the rolling buffer in a persistent memory for editing, whereby the at least the portion of the audio-video streams is saved after it has already been presented; and
provide ana graphical user interface to edit the video frames and the at least two streams of audio from the portion of the audio-video streams stored in the persistent memory, wherein the graphical user interface to edit provides an edited clip.
17 . The computer-readable storage medium of claim 16 , wherein the instructions further configure the computer to:
receive a selection of a first audio stream of the at least two streams of audio for editing, wherein respective audio streams within the at least two streams of audio can be independently edited.
18 . The computer-readable storage medium of claim 16 , wherein the storing the video frames in the rolling buffer further comprises:
encode the video frames using a first encoder, wherein the first encoder dynamically selects a keyframe interval.
19 . The computer-readable storage medium of claim 16 , wherein the instructions further configure the computer to:
determine that the video frames and the at least two streams of audio in the rolling buffer are likely to be selected for use in the clip;
dynamically increase a size of the rolling buffer to avoid deletion of portions of the rolling buffer.
20 . The computer-readable storage medium of claim 16 , wherein the instructions further configure the computer to:
encode the video frames, where the encoding the video frames includes:
determine an encoding pipeline that is optimized to a client device in which a client application is executing, the determining comprising:
identify whether a hardware encoder is available;
select the hardware encoder when the hardware encoder is available.