Interactive streaming media production tool using communication optimization
View Patent ↗The invention provides interactive streaming mixed media production. A mixed media content creation tool includes a resource-computation element that computes the likely resource load to be imposed on a server, communication link, and client device, when sent by the server to the client and presented to a user at the client. The resource-computation element is responsive to amounts of communication bandwidth available using the communication link, presentation buffer space and computing resources available at the client, and other measures of resource load. The resource-computation element attempts to optimize resource usage by the mixed media content. The system also includes a presentation element that responds to the resource-computation element, and to user requests, and interactively presents measures of projected resource usage by the mixed media content, comparisons of that projected resource usage with available resources, and comparisons of that projected resource usage before and after optimization.
1. A method for calculating a total necessary communication bandwidth and transmission time corresponding to a stream of mixed media including multiple media streams, including steps of
receiving inputs at a fitter;
determining a feasibility of sending said mixed media in light of said inputs;
dividing data comprising said mixed media stream into data packets;
determining a minimal send time for each of said data packets, wherein said minimal send time is the earliest time each of said packets may be sent with respect to a maximum permitted send ahead time imposed by the network; and
assigning a send time to each said data packet, the assigned time being no earlier than the determined minimal send time;
the method including steps of converting raw data present in the stream of mixed media from a first format to a second format.
2. A method as in claim 1 , wherein said inputs include a maximum packet size and an available channel bandwidth as a function of time, a time offset of for each said media stream, a maximum decoding buffer size, a maximum permitted send ahead time, and an overhead of transport protocol per packet.
3. A method as in claim 1 , including a step of notifying a content creator of the feasibility of sending said mixed media.
4. A method as in claim 1 , wherein said step of dividing the data is performed by a fitter.
5. A method as in claim 1 , wherein said step of determining the minimal send time includes simulating a decoding buffer.
6. A method as in claim 5 , wherein said minimal send time is set to a maximum of (1) a date stamp of a current said packet minus the maximum permitted send ahead time or (2) the date stamp of the packet most recently removed from said decoding buffer.
7. A method as in claim 1 , wherein said step of assigning said send time is performed starting with said data packets associated with an end of a performance of said mixed media and moving toward said data packets associated with a beginning of the performance of said mixed media.
8. A method as in claim 1 , wherein said send time is set to a nominal send time minus a size of said packet divided by an available bandwidth at said nominal send time.
9. A method comprising steps of
importing a media stream;
encoding said media stream;
computing a bit rate associated with one or more elements included in said media stream;
generating a graphical image responsive to said step of computing, wherein said graphical image includes a time display and a set of information on bit rate usage;
updating said time display and said set of information; and
exporting said media stream;
the method including steps of converting raw data present in the stream of mixed media from a first format to a second format.
10. A method as in claim 9 , wherein said step of encoding includes steps of converting a format associated with said media stream to an MPEG 4 format.
11. A method as in claim 9 , wherein said step of computing includes computing a bit rate that is responsive to a buffer space at an end user, a communication link or a server.
12. A method as in claim 9 , wherein said time display includes a timeline, comprising a life cycle associated with said one or more elements.
13. A method as in claim 9 , wherein said step of updating is responsive to one or more changes made in said media stream.
14. A method including steps of
composing a mixed media including one or more objects;
calculating a total necessary communication bandwidth and transmission time corresponding to said mixed media;
modifying said mixed media in response to said step of calculating, wherein said step of modifying changes an order of transmission or a specific transmission time for said objects; and
displaying a graphical representation of optimization before and after the step of calculating;
the method including steps of converting raw data present in the stream of mixed media from a first format to a second format.
15. A method as in claim 14 , wherein said step of modifying includes adding, deleting, or changing attributes associated with said object.
16. A method as in claim 14 , wherein said step of displaying includes displaying a set of different media taking part in a global resource usage.
17. A method as in claim 14 , wherein step of displaying includes providing a visual feedback of a download and decoding time required to transfer and decode said mixed media from a server to a player.
18. A system that calculates a total necessary communication bandwidth and transmission time corresponding to a stream of mixed media including multiple media streams, comprising:
input and output elements disposed to receive inputs and to send mixed media;
a processor and memory that stores instructions that, when executed by said processor, cause the system to effect the steps of any one of claims 1 - 8 and 9 - 17 .
19. A computer program product comprising program code means that, when executed on a computer system, causes the computer system to effect the steps of any one of claims 1 - 8 and 9 - 17 .
20. A memory that stores instructions that, when executed by a processor of a system capable of calculating communication bandwidth and transmission time corresponding to a stream of mixed media with multiple media streams, cause the system to effect the steps of any one of claims 1 - 8 and 9 - 17 .