Transparent insertion of multimedia content in real-time streaming sessions
An example embodiment of the present invention provides a process for inserting a source media stream into a target media stream, which media streams might be in container file format. The process opens and reads into memory structural information from a target media stream and structural information from a source media stream. The process then finds the nearest prior key frame to an insertion time and splits a chunk of media samples if the key frame's sample is not at the beginning of a chunk. The process adjusts the structural information for both streams in order to effect the mid-roll insertion of the source media stream into the target media stream. The process then writes onto an output media stream the adjusted structural information for the source and target media streams and then pipes onto the output media stream the media data for the source and target media streams.
1. A method, comprising
receiving, at an insertion server, a request to access a first media stream from a mobile device comprising a client, wherein the first media stream is hosted by a first streaming server;
identifying, at the insertion server, a second media stream for insertion into the first media stream and an insertion point for the second media stream, wherein the second media stream is hosted by a second streaming server; and
transparently inserting, via the insertion server, the second media stream into the first media stream by:
proxying, via the insertion server, a first streaming session between the client and the first streaming server;
proxying, via the insertion server, a second streaming session between the client and the second streaming server; and
modifying, at the insertion server, stream parameter information wirelessly transmitted by the mobile device from the client and native media stream parameters transmitted by the first and second streaming servers in connection with the first and second streaming sessions to combine, via the insertion server and relative to the client, the first and second streaming sessions as a single streaming session,
wherein the native media stream parameters comprise information describing the first media stream received at the insertion server from the first streaming server and information describing the second media stream received at the insertion server from the second streaming server; and
determining, at the insertion server, whether the first media stream and the second media stream are compatible for insertion based upon, at least in part, the native media stream parameters; and combining, at the insertion server, at least part of the information describing the first media stream received at the insertion server from the first streaming server and at least part of the information describing the second media stream received at the insertion server from the second streaming server by aggregating temporal parameters of the first media stream and the second media stream.
2. The method of claim 1 wherein the first and second media streams are audio streams.
3. The method of claim 1 wherein the first and second media streams are video streams.
4. The method of claim 1 wherein the first and second streaming sessions each comprise a data connection, a control connection and a command connection.
5. The method of claim 1 wherein the native media stream parameters comprises packet sequence number information.
6. The method of claim 1 wherein the native media stream parameters comprises stream timing information.
7. The method of claim 1 wherein the native media stream parameters comprises one or more stream source identifiers.
8. The method of claim 1 further comprising controlling messages being sent to or from the second media server using a state machine implemented on the insertion server.
9. The method of claim 8 , further comprising buffering Real Time Streaming Protocol commands received from the client until a play state of the state machine occurs.
10. The method of claim 1 wherein the client is a multimedia player.
11. An apparatus, comprising
one or more network interfaces;
one or more processors;
a state machine,
a media insertion module comprising computer-readable instructions stored in a physical medium, the computer-readable instructions operative, when executed, to cause the one or more processors to:
deploy an ad campaign to a mobile device;
receive, at an insertion server comprising the state machine, a request to access a first media stream from the mobile device, wherein the first media stream is hosted by a first streaming server;
identify, at the insertion server, a second media stream for insertion into the first media stream and an insertion point for the second media stream, wherein the second media stream is hosted by a second streaming server; and
transparently insert, at the insertion server, the second media stream into the first media stream by:
proxying, via the insertion server, a first streaming session between the mobile device and the first streaming server;
proxying, via the insertion server, a second streaming session between the mobile device and the second streaming server; and
modifying, at the insertion server, stream parameter information wirelessly transmitted by the mobile device and native media stream parameters transmitted by the first and second streaming servers in connection with the first and second streaming sessions to combine, via the insertion server and relative to the mobile device, the first and second streaming sessions as a single streaming session comprising an ad of the ad campaign, wherein states of the state machine control how control messages are forwarded between the mobile device and the first streaming server and the second streaming server,
wherein the native media stream parameters comprise information describing the first media stream received at the insertion server from the first streaming server and information describing the second media stream received at the insertion server from the second streaming server; and
determining, at the insertion server, whether the first media stream and the second media stream are compatible for insertion based upon, at least in part, the native media stream parameters; and combining, at the insertion server, at least part of the information describing the first media stream received at the insertion server from the first streaming server and at least part of the information describing the second media stream received at the insertion server from the second streaming server by aggregating temporal parameters of the first media stream and the second media stream.
12. A method, comprising
receiving, at an insertion server, a request to access a first media stream from a media stream client installed on a mobile device, wherein the first media stream is hosted by a first streaming server;
executing a state machine implemented on the insertion server;
forwarding a control message between the mobile device and the first streaming server and a second streaming server using the state machine;
identifying, at an insertion server, a second media stream for insertion into the first media stream and an insertion point for the second media stream, wherein the second media stream is hosted by the second streaming server; and
transparently inserting, via the insertion server, the second media stream into the first media stream by:
proxying, via the insertion server, a first streaming session between the media stream client and the first streaming server;
proxying, via the insertion server, a second streaming session between the media stream client and the second streaming server; and
modifying, at the insertion server, stream parameter information wirelessly transmitted by the media stream client using the mobile device and native media stream parameters transmitted by the first and second streaming servers in connection with the first and second streaming sessions to combine, via the insertion server and relative to the media stream client, the first and second streaming sessions as a single streaming session, wherein the stream parameter information comprises temporal information,
wherein the native media stream parameters comprise information describing the first media stream received at the insertion server from the first streaming server and information describing the second media stream received at the insertion server from the second streaming server; and
determining, at the insertion server, whether the first media stream and the second media stream are compatible for insertion based upon, at least in part, the native media stream parameters; and combining, at the insertion server, at least part of the information describing the first media stream received at the insertion server from the first streaming server and at least part of the information describing the second media stream received at the insertion server from the second streaming server by aggregating temporal parameters of the first media stream and the second media stream.
13. The method of claim 12 wherein states of the state machine implemented on the insertion server comprises a match state, a play state, and a switch state.
14. The method of claim 12 , wherein the temporal parameters of the first media stream and the second media stream include a length of the first media stream and a length of the second media stream.