IP Library Granted Patent US 7,992,165
Granted Patent B1
US 7,992,165 · App. 11/674,570 · Granted Aug 2, 2011

Insertion of digital media

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,992,165
App. No.
11/674,570
Granted
Aug 2, 2011
Kind
B1
Abstract

An example embodiment of the present invention provides a process for inserting a source media stream into a target media stream. The process opens and reads into memory structural information from a target media stream. The process then opens and reads into memory structural information from a source media stream. The process adjusts the structural information for both streams in order to effect the insertion of the source media stream into the target media stream. In the example embodiment, this adjustment avoids the use of temporary output storage. The process then writes onto an output media stream the adjusted structural information for the source and target media streams and then pipes onto the output media stream the media data for the source and target media streams. In particular embodiments, a container file format is used for the source media stream, target media stream, and output media stream.

Claims (78)

1. A method, comprising:

receiving a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

receiving a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

reading the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

reading the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

writing onto an output media stream the adjusted first and second media streams; and

piping onto the output media stream media data of the first and second adjusted media streams, wherein piping onto the output media stream media data of the first and second media streams comprises utilizing file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

2. A method as in claim 1 , further comprising prenormalizing one or more of the first media stream and the second media stream prior to the adjusting.

3. A method as in claim 2 , wherein prenormalizing comprises transcoding one or more of the first and second media streams into common compatible format.

4. A method as in claim 2 , wherein prenormalizing comprises, for one or more of the first and second media streams, resequencing media samples in the stream to generate a linear sequence of media samples.

5. A method as in claim 1 , wherein the method is performed by an insertion server.

6. A method as in claim 1 , wherein the first media stream comprises an advertisement and the second media stream comprises content.

7. A method as in claim 1 , wherein the first container file format is an MP4 file format.

8. A method as in claim 1 , wherein writing onto the output media stream is concurrent with one or more of receiving the first media stream and receiving the second media stream.

9. A method as in claim 1 , wherein the output media stream is transmitted to a mobile device for consumption by the mobile device.

10. A method as in claim 1 , wherein adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream comprises one or more of:

adjusting the duration in the second stream header to be a sum of an original duration of the first media stream and an original duration of the second media stream;

launching a nested for-loop that iterates over each first track in the first media stream;

launching a nested for-loop that iterates over each second track in the second media stream;

adjusting the duration in each of one or more of the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers to account for the new media length;

adjusting the sample count in the time-to-sample structure in one or more of the first and second sample tables;

adding additional samples to the sync-sample structure in one or more of the first and second sample tables;

adjusting the sample-to-chunk structure in one or more of the first and second sample tables to account for the additional samples;

adding the additional samples to the sample size structure in one or more of the first and second sample tables;

adjusting the chunk offset structure in one or more of the first and second sample tables to account for the additional samples and adding the new chunks to the structure;

recalculating the size of each of one or more of the first track headers, second track headers, first edit lists, second edit lists, first media headers, second media headers, first sample tables, and second sample tables before the first or second nested for-loops end;

recalculating a total size of all of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables; and

readjusting a chunk-to-offset table based on the recalculation of the total size.

11. The method of claim 1 , wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

12. The method of claim 1 , wherein piping onto the output media stream media data of the first and second adjusted media streams comprises:

concurrently reading the media data from the first adjusted media stream and the second adjusted media stream and writing the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.

13. An apparatus, comprising logic encoded in one or more persistent storage media for execution and when executed operable to:

receive a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

receive a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

read the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

read the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

adjust one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in both streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

write onto an output media stream the adjusted first and second streams; and

pipe onto the output media stream media data of the first and second adjusted media streams, wherein the logic operable to pipe onto the output media stream media data of the first and second media streams comprises logic operable to utilize file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

14. An apparatus as in claim 13 , wherein the logic is further operable to prenormalize one or more of the first media stream and the second media stream.

15. An apparatus as in claim 13 , wherein the encoded logic is executed by an insertion server.

16. An apparatus as in claim 13 , wherein the first media stream comprises an advertisement and the second media stream comprises content.

17. An apparatus as in claim 13 , wherein the first container file format is an MP4 file format.

18. An apparatus as in claim 13 , wherein the logic operable to write onto the output media stream, receive the first media stream, and receive the second media stream is operable to write onto the output media stream concurrently with one or more of receiving the first media stream and receiving the second media stream.

19. An apparatus as in claim 13 , wherein the logic operable to adjust one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in the first and second media streams in order to effect the insertion of the first media stream into the second media stream comprises logic operable to perform one or more of:

adjust the duration in the second stream header to be a sum of an original duration of the first media stream and an original duration of the second media stream;

launch a nested for-loop that iterates over each first track in the first media stream;

launch a nested for-loop that iterates over each second track in the second media stream;

adjust the duration in each of one or more of the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers to account for the new media length;

adjust the sample count in the time-to-sample structure in one or more of the first and second sample tables;

add additional samples to the sync-sample structure in one or more of the first and second sample tables;

adjust the sample-to-chunk structure in one or more of the first and second sample tables to account for the additional samples;

add the additional samples to the sample size structure in one or more of the first and second sample tables;

adjust the chunk offset structure in one or more of the first and second sample tables to account for the additional samples and adding the new chunks to the structure;

recalculate the size of each of one or more of the first track headers, second track headers, first edit lists, second edit lists, first media headers, second media headers, first sample tables, and second sample tables before the first or second nested for-loops end;

recalculate a total size of all of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables; and

readjust a chunk-to-offset table based on the recalculation of the total size.

20. The apparatus of claim 13 , wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

21. The apparatus of claim 13 , wherein pipe onto the output media stream media data of the first and second adjusted media streams comprises:

concurrently read the media data from the first adjusted media stream and the second adjusted media stream and write the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.

22. An apparatus, comprising:

means for receiving a first media stream formatted into a first container file format and encoded using a first codec, the first media stream comprising a first stream header and one or more first tracks, each first track comprising a first track header, one or more first edit lists, and one or more first media, each first media comprising a first media header and one or more first sample tables specifying locations and durations of sample data within the first media stream;

means for receiving a second media stream formatted into the first container file format and encoded using the first codec, the second media stream comprising a second stream header and one or more second tracks, each second track comprising a second track header, one or more second edit lists, and one or more second media, each second media comprising a second media header and one or more second sample tables specifying locations and durations of sample data within the second media stream;

means for reading the first stream header, the first track headers, the first edit lists, the first media headers, and the first sample tables;

means for reading the second stream header, the second track headers, the second edit lists, the second media headers, and the second sample tables;

means for adjusting one or more of the first stream header, the second stream header, the first track headers, the second track headers, the first edit lists, the second edit lists, the first media headers, the second media headers, the first sample tables, and the second sample tables in both streams in order to effect the insertion of the first media stream into the second media stream without removing any content of the second media stream;

means for writing onto an output media stream the adjusted first and second media streams; and

means for piping onto the output media stream media data of the first and second adjusted media streams, wherein the means for piping onto the output media stream media data of the first and second media streams comprise means for utilizing file channels that operate on temporary copies of the first and second media streams, wherein each file channel is operative to establish a buffer directly mapped to file contents in the first and second media streams using memory-mapped I/O.

23. The apparatus of claim 22 , wherein:

the first stream header, the first track headers, the first media headers, the second stream header, the second track headers, and the second media headers are adjusted and written to a temporary memory and then written to the output media stream; and

the media data of the first media stream and the second media stream are directly piped onto the output media stream without using any temporary memory.

24. The apparatus of claim 22 , wherein means for piping onto the output media stream media data of the first and second adjusted media streams comprises:

means for concurrently reading the media data from the first adjusted media stream and the second adjusted media stream and writing the media data to the output media stream so that no temporary output storage is used while piping onto the output media stream the media data of the first and second adjusted media streams.

Assignments (22)
RELEASE OF SECURITY INTEREST IN PATENT COLLATERAL Recorded Jul 22, 2021
From: CORTLAND CAPITAL MARKET SERVICES LLC
To: VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC; VIVIAL INC.
Reel/Frame 056946/0459 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jun 17, 2021
From: PNC BANK, NATIONAL ASSOCIATION
To: VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC; VIVIAL INC.
Reel/Frame 056618/0051 →
SECURITY INTEREST Recorded Dec 27, 2018
From: VIVIAL INC.; VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC
To: CORTLAND CAPITAL MARKET SERVICES LLC, AS COLLATERAL AGENT
Reel/Frame 047863/0520 →
RELEASE OF SECURITY INTEREST Recorded Dec 27, 2018
From: TPG SPECIALTY LENDING, INC., AS COLLATERAL AGENT
To: VIVIAL, INC.; VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC
Reel/Frame 047860/0351 →
RELEASE OF SECURITY INTEREST Recorded Oct 17, 2017
From: HERITAGE BANK OF COMMERCE
To: MGAGE, LLC; OUTSPOKEN, LLC
Reel/Frame 043882/0047 →
SECURITY INTEREST Recorded Oct 4, 2017
From: VIVIAL INC.; VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC
To: PNC BANK, NATIONAL ASSOCIATION
Reel/Frame 044116/0896 →
GRANT OF A SECURITY INTEREST -- PATENTS Recorded Oct 2, 2017
From: VIVIAL INC.; VIVIAL HOLDINGS LLC; VIVIAL MEDIA LLC; VIVIAL MOBILE LLC; VIVIAL NETWORKS LLC; MGAGE, LLC
To: TPG SPECIALTY LENDING, INC., AS COLLATERAL AGENT
Reel/Frame 044131/0830 →
SECURITY INTEREST Recorded Feb 24, 2017
From: MGAGE, LLC; OUTSPOKEN, LLC
To: HERITAGE BANK OF COMMERCE
Reel/Frame 041377/0123 →
RELEASE OF SECURITY INTEREST Recorded Sep 1, 2015
From: U.S. BANK NATIONAL ASSOCIATION
To: MGAGE, LLC
Reel/Frame 036471/0442 →
SECURITY INTEREST Recorded Aug 27, 2015
From: MGAGE, LLC
To: PNC BANK, NATIONAL ASSOCIATION
Reel/Frame 036436/0127 →
SECURITY INTEREST Recorded Mar 24, 2014
From: MGAGE, LLC
To: U.S. BANK, NATIONAL ASSOCIATION
Reel/Frame 032511/0597 →
CHANGE OF NAME Recorded Mar 5, 2014
From: GSO MMBU ACQUISITION LLC
To: MGAGE, LLC
Reel/Frame 032390/0163 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 4, 2014
From: VELTI INC.
To: GSO MMBU ACQUISITION LLC
Reel/Frame 032346/0301 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jan 27, 2014
From: U.S. BANK, NATIONAL ASSOCIATION (AS SUCCESSOR TO HSBC BANK USA, NATIONAL ASSOCIATION)
To: VELTI INC.; VELTI MOBILE PLATFORMS LIMITED
Reel/Frame 032127/0228 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jan 27, 2014
From: U.S. BANK, NATIONAL ASSOCIATION
To: VELTI INC.; AIR2WEB, INC.
Reel/Frame 032127/0326 →
CORRECTIVE ASSIGNMENT TO CORRECT THE DATE OF EXECUTION OF DOCUMENT PREVIOUSLY RECORDED ON REEL 031795 FRAME 0612. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY AGREEMENT. Recorded Dec 20, 2013
From: VELTI INC.; AIR2WEB, INC.
To: U.S. BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT
Reel/Frame 031866/0943 →
NOTICE OF CHANGE OF AGENT AND ASSIGNMENT OF SECURITY INTEREST IN PATENT COLLATERAL Recorded Dec 12, 2013
From: HSBC BANK USA, NATIONAL ASSOCIATION
To: U.S. BANK, NATIONAL ASSOCIATION
Reel/Frame 031808/0504 →
SECURITY AGREEMENT Recorded Dec 11, 2013
From: VELTI INC.; AIR2WEB, INC.
To: U.S. BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT
Reel/Frame 031795/0612 →
CHANGE OF NAME Recorded Dec 11, 2013
From: VELTI USA, INC.
To: VELTI INC.
Reel/Frame 031759/0676 →
SECURITY AGREEMENT Recorded Aug 13, 2012
From: VELTI INC., A DELAWARE CORPORATION, AS GRANTOR
To: HSBC BANK USA, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT
Reel/Frame 028779/0253 →
CHANGE OF NAME Recorded Jul 26, 2012
From: AD INFUSE, INC.
To: VELTI USA, INC.
Reel/Frame 028647/0785 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2007
From: LUDEWIG, CARL; HUANG, ALEX XUEYUAN
To: AD INFUSE, INC.
Reel/Frame 018887/0150 →