Manipulation of media streams in the compressed domain
View Patent ↗An integrated circuit receives a compressed input stream having a first compression format. A media processing module converts the compressed input stream to an intermediary compression format for processing without fully decompressing the compressed input stream. After processing, a compressed output stream having a second compression format is generated from the intermediary compression format. Processing is dynamically adjusted responsive to changing network conditions. Optionally, the integrated circuit can receive live, raw video, partially encode it into the intermediary compression format, process it with the media process module as well as take the intermediary compression format, decode and output the live, raw video.
1. An integrated circuit, comprising:
a frame converter configured to:
extract motion parameters from a compressed media stream in a first compressed format, the first compressed format including input frames that are interframe compressed and intraframe compressed; and
convert the input frames in the first compressed format into an intermediate media stream in a second compressed format, the second compressed format including intermediate frames that are intraframe compressed but not interframe compressed, the second compressed format not including frames that are interframe compressed; and
an image processor configured to perform at least one of removing mosquito noise, removing of blockiness or deinterlacing on the intermediate media stream in a compressed domain without decompressing the intermediate media stream to generate a processed media stream; and
an encoder separate from the image processor and configured to encode the processed media stream to an output media stream in a third compressed format by using the processed media stream and modified motion parameters derived from the extracted motion parameters.
2. The integrated circuit of claim 1 , further comprising a scaler to process frames in the intermediate media stream from a first resolution to a second resolution.
3. The integrated circuit of claim 1 , wherein the encoder is configured to transrate the intermediate media stream to generate the output stream.
4. The integrated circuit of claim 1 , wherein the first compressed format is different from the third compressed format.
5. The integrated circuit of claim 1 , wherein the first compressed format is a format selected from one of MPEG 2, MPEG 1, H.263, WMV-9, VC-1, DV, AVS, MPEG 4, JPEG, and H.264.
6. A method for converting media streams in an integrated circuit, the method comprising:
receiving a compressed media stream in a first compressed format, the first compressed format including input frames that are interframe compressed and intraframe compressed;
converting input frames in the first compressed format into an intermediate media stream in a second compressed format, the second compressed format including intermediate frames that are intraframe compressed but not interframe compressed, the second compressed format not including frames that are interframe compressed;
performing at least one of removing mosquito noise, removing of blockiness or deinterlacing on the intermediate media stream in a compressed domain without decompressing the intermediate media stream to generate a processed media stream; and
converting the processed media stream into an output stream in a third compressed format by using the processed media stream and modified motion parameters derived from the extracted motion parameters, wherein converting the processed media stream is a process separate from performing the at least one of removing mosquito noise, removing of blockiness or deinterlacing.
7. The method of claim 6 , further comprising processing frames in the intermediate media stream from a first resolution to a second resolution.
8. The method of claim 6 , further comprising transrating the intermediate media stream to generate the processed media stream.
9. The method of claim 6 , wherein the first compressed format is a format selected from one of MPEG 2, MPEG 1, H.263, WMV-9, VC-1, DV, AVS, MPEG 4, JPEG, and H.264.
10. The integrated circuit of claim 1 , wherein the modified motion parameters are derived from the extracted motion parameters by modifying sizes or definitions of the extracted motion parameters.
11. The integrated circuit of claim 10 , wherein the modified motion parameters comprise motion vectors.
12. The integrated circuit of claim 1 , wherein the image processor is further configured to combine the intermediate media stream with another media stream to generate the processed media stream.
13. The integrated circuit of claim 1 , further comprising a network processing unit configured to detect a condition of a network via which the output media stream is transmitted, one or more parameters associated with the image enhancement determined based on the network condition.
14. The integrated circuit of claim 13 , wherein the image process determines a transcaling parameter based on the network condition.
15. The method of claim 6 , wherein the modified motion parameters are derived from the extracted motion parameters by modifying sizes or definitions of the extracted motion parameters.
16. The method of claim 15 , wherein the modified motion parameters comprise motion vectors.
17. The method of claim 6 , further comprising combining the intermediate media stream with another media stream to generate the processed media stream.
18. The method of claim 6 , further comprising:
detecting a condition of a network via which the output media stream is transmitted; and
determining one or more parameters associated with the image enhancement based on the network condition.
19. The method of claim 18 , wherein determining one or more parameters comprises determining a transcaling parameter based on the network condition.
20. A non-transitory computer readable storage medium storing instructions thereon, the instructions when executed by a processor, causing the processor to:
receive a compressed media stream in a first compressed format, the first compressed format including input frames that are interframe compressed and intraframe compressed;
convert input frames in the first compressed format into an intermediate media stream in a second compressed format, the second compressed format including intermediate frames that are intraframe compressed but not interframe compressed, the second compressed format not including frames that are interframe compressed;
perform at least one of removing mosquito noise, removing of blockiness or deinterlacing on the intermediate media stream in a compressed domain without decompressing the intermediate media stream to generate a processed media stream; and
convert the processed media stream into an output stream in a third compressed format by using the processed media stream and modified motion parameters derived from the extracted motion parameters, wherein converting the processed media stream is a process separate from performing the at least one of removing mosquito noise, removing of blockiness or deinterlacing.
21. The computer readable storage medium of claim 20 , further comprising instruction to process frames in the intermediate media stream from a first resolution to a second resolution.