Content classification for multimedia processing
An apparatus and method for processing multimedia data, such as, for example, video data, audio data, or both video and audio data for encoding utilizing a determined content classification is claimed. Processing the multimedia data includes determining complexity of multimedia data, classifying the multimedia data based on the determined complexity, and, determining a bit rate for encoding the multimedia data based on its classification. The complexity can include a spatial complexity component and a temporal complexity component of the multimedia data. The multimedia data is classified using content classifications, which are based on a visual quality value for viewing multimedia data, using the spatial complexity, the temporal complexity, or both the spatial complexity and temporal complexity.
1. A method of processing multimedia data comprising:
retrieving, from a memory device, preprocessed information related to the multimedia data;
determining a complexity of the multimedia data from the retrieved information, wherein determining the complexity comprises determining spatial complexity and temporal complexity of the multimedia data; and
classifying the multimedia data based on the determined complexity, wherein classifying the multimedia data comprises:
associating the spatial complexity with a texture value;
associating the temporal complexity with a motion value; and
assigning a content classification to the multimedia data based on the texture value and the motion value;
determining a content classification curve based on the assigned content classification; and
based on a target quality level for the multimedia data and the determined content classification curve, determining a bit rate for the multimedia data.
2. The method of claim 1 , wherein determining spatial complexity comprises determining at least one human visual sensitivity value for at least one block of the multimedia data.
3. The method of claim 1 , wherein determining spatial complexity comprises determining at least one contrast ratio value for at least one block of the multimedia data.
4. The method of claim 1 , wherein determining temporal complexity comprises determining at least one motion vector (MV) for at least one block of the multimedia data.
5. The method of claim 1 , wherein determining temporal complexity further comprises associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.
6. The method of claim 1 , wherein determining temporal complexity comprises determining at least one frame difference value for at least one block of the multimedia data.
7. The method of claim 6 , wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.
8. The method of claim 1 , wherein classifying the multimedia data comprises using a plurality of content classifications.
9. An apparatus for processing multimedia data comprising:
means for retrieving, from a memory device, preprocessed information related to the multimedia data;
means for determining a complexity of the multimedia data from the retrieved information, wherein the means for determining the complexity comprises means for determining spatial complexity and temporal complexity of the multimedia data; and
means for classifying the multimedia data content based on the determined complexity, wherein the means for classifying the multimedia data comprises:
means for associating the spatial complexity with a texture value;
means for associating the temporal complexity with a motion value; and
means for assigning a content classification to the multimedia data based on the texture value and the motion value;
means for determining a content classification curve based on the assigned content classification;
means for determining a bit rate for the multimedia data based on a target quality level for the multimedia data and the determined content classification curve.
10. The apparatus of claim 9 , wherein the means for determining spatial complexity comprises means for determining at least one human visual sensitivity value for at least one block of the multimedia data.
11. The apparatus of claim 9 , wherein the means for determining spatial complexity comprises means for determining at least one contrast ratio value for at least one block of the multimedia data.
12. The apparatus of claim 9 , wherein the means for determining temporal complexity comprises means for determining at least one motion vector (MV) for at least one block of the multimedia data.
13. The apparatus of claim 12 , wherein the means for determining temporal complexity further comprises means for determining a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.
14. The apparatus of claim 9 , wherein the means for determining temporal complexity comprises means for determining at least one frame difference value determined for at least one block of the multimedia data.
15. The apparatus of claim 14 , wherein the at least one frame difference value is a SAD_MV value determined for at least one block of the multimedia data.
16. The apparatus of claim 9 , wherein means for classifying the multimedia data content comprises a plurality of means for content classifications.
17. A non-transitory computer readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to:
determine complexity of multimedia data, wherein the one or more processors determine the complexity by determining spatial complexity and temporal complexity of the multimedia data;
classify multimedia data based on the determined complexity, wherein classifying the multimedia data comprises:
associating the spatial complexity with a texture value;
associating the temporal complexity with a motion value; and
assigning a content classification to the multimedia data based on the texture value and the motion value;
determine a content classification curve based on the assigned content classification; and
based on a target quality level for the multimedia data and the determined content classification curve, determine a bit rate for the multimedia data.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors determine temporal complexity by determining at least one human visual sensitivity value for at least one block of the multimedia data.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors determine temporal complexity by determining at least one contrast ratio value for at least one block of the multimedia data.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors determine temporal complexity by determining at least one motion vector (MV) for at least one block of the multimedia data.
21. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors determine temporal complexity by associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.
22. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors determine temporal complexity by determining at least one frame difference value for at least one block of the multimedia data.
23. The non-transitory computer-readable storage medium of claim 22 , wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.
24. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more processors classify the multimedia data by using a plurality of content classifications.
25. A device for processing multimedia data, the device comprising:
a memory;
one or more processors configured to:
retrieve, from the memory, preprocessed information related to the multimedia data;
determine a complexity of the multimedia data from the retrieved information, wherein determining the complexity comprises determining spatial complexity and temporal complexity of the multimedia data;
classify the multimedia data based on the determined complexity, wherein classifying the multimedia data comprises:
associating the spatial complexity with a texture value;
associating the temporal complexity with a motion value; and
assigning a content classification to the multimedia data based on the texture value and the motion value;
determine a content classification curve based on the assigned content classification; and
based on a target quality level for the multimedia data and the determined content classification curve, determine a bit rate for the multimedia data.
26. The device of claim 25 , wherein the one or more processors determine temporal complexity by determining at least one human visual sensitivity value for at least one block of the multimedia data.
27. The device of claim 25 , wherein the one or more processors determine temporal complexity by determining at least one contrast ratio value for at least one block of the multimedia data.
28. The device of claim 25 , wherein the one or more processors determine temporal complexity by determining at least one motion vector (MV) for at least one block of the multimedia data.
29. The device of claim 25 , wherein the one or more processors determine temporal complexity by associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.
30. The device of claim 25 , wherein the one or more processors determine temporal complexity by determining at least one frame difference value for at least one block of the multimedia data.
31. The device of claim 30 , wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.
32. The device of claim 25 , wherein the one or more processors classify the multimedia data by using a plurality of content classifications.