Method and apparatus of managing visual content
A system and method is provided for managing visual content. In one instance, an exemplary method includes receiving a stream of video fingerprints derived in a fingerprint generator by an irreversible data reduction process, from respective temporal regions within a particular visual content stream and at a fingerprint processor that is physically separate from the fingerprint generator via a communication network. The fingerprints are processed in the fingerprint processor to generate metadata which is not directly encoded in the fingerprints. Processing of the fingerprints includes windowing the stream of fingerprints with a time window, deriving frequencies of occurrence of particular fingerprint values or ranges of fingerprint values within each time window, determining statistical moments or entropy values of said frequencies of occurrence, comparing said statistical moments or entropy values with expected values for particular types of content, and generating metadata representing the type of the visual content.
1. A system for managing audio visual content, the system comprising:
a window selector configured to set at least one time window for a received audio visual stream having a plurality of content fingerprints derived by a fingerprint generator using an irreversible data reduction process from respective temporal regions within the audio visual stream;
a separator configured to separate out at least one audio or video component from the content fingerprints in the at least one time window, with the at least one audio or video component corresponding to a content characteristic of the audio visual stream;
a test signal detector configured to generate at least one fingerprint value, respectively, of the separated at least one audio or video component by executing a statistical analysis operation that comprises at least one of comparing the at least one audio or video component to a predetermined threshold and comparing the at least one audio or video component to a range of known values;
a histogram generator configured to generate a histogram based on a frequency of occurrences of the generated at least one fingerprint value occurring in the at least one time window of the audio visual stream; and
an audio visual data classifier configured to generate statistical metadata by comparing the frequency of occurrences set forth in the generated histogram to at least one known type of audio visual content,
wherein the generated statistical metadata configures an automatic control system for distributing the audio visual stream.
2. The system according to claim 1 , further comprising an entropy value generator configured to derive entropy values for each generated histogram, and wherein the audio visual data classifier is configured to generate the statistical metadata that represents a type of the audio visual content based on the derived entropy values.
3. The system according to claim 1 , wherein the generated statistical metadata comprises one or more of a mean, a variance, a skew and a kurtosis of the frequency of occurrences.
4. The system according to claim 1 , wherein the audio visual content comprises a video stream of video frames, with at least one fingerprint generated for every frame in the video stream.
5. The system according to claim 1 ,
wherein the at least one audio or video component is an audio sample component, and
wherein the audio visual data classifier comprises an audio level detector configured to generate the statistical metadata as audio level metadata indicating a loudness level of the audio visual stream when a sustained number of occurrences of the at least one audio or video component are below a low-level threshold for a fingerprint sequence of the at least one time window of the audio visual stream.
6. The system according to claim 1 , wherein the audio visual data classifier is further configured to generate the statistical metadata as a weighted sum of outputs from a plurality of different comparison functions, wherein respective weights and functions are previously selected by a training process where candidate sets of comparison functions are applied iteratively to sets of statistical data and entropy data derived from an analysis of fingerprint data from known types of audio visual content.
7. The system according to claim 1 , wherein the audio visual data classifier comprises a shot-change detector configured to identify isolated high values of the content fingerprints by comparing respective value differences between a first fingerprint and preceding and succeeding content fingerprints with a threshold, and the audio visual data classifier is configured to generate shot-change metadata when the respective value differences are above a predetermined threshold.
8. A system for managing audio visual content, the system comprising:
a separator configured to separate out at least one audio or video component from at least one content fingerprint in a received audio visual fingerprint stream, with the at least one audio or video component corresponding to a content characteristic of an audio visual stream associated with the received audio visual fingerprint stream;
a test signal detector configured to generate at least one fingerprint value, respectively, of the separated at least one audio or video component by executing a statistical analysis operation of the at least one audio or video component;
a histogram generator configured to generate a histogram based on a frequency of occurrences of the generated at least one fingerprint value within a fingerprint sequence of the audio visual fingerprint stream; and
an audio visual data classifier configured to generate control metadata by comparing the frequency of occurrences set forth in the generated histogram to at least one known type of audio visual content;
wherein the at least one audio or video component is an audio sample component, and the test signal detector comprises an audio level detector configured to generate audio level metadata indicating a loudness level of the audio visual stream when a sustained number of occurrences of the audio sample component within the fingerprint sequence of the audio visual stream are below a low-level threshold;
wherein the generated control metadata configures an automatic control system for distributing the audio visual stream.
9. The system according to claim 8 , further comprising a window selector configured to set at least one time window for the received audio visual fingerprint stream having the at least one content fingerprint.
10. The system according to claim 8 , wherein the statistical analysis operation comprises at least one of comparing the at least one audio or video component to a predetermined threshold and comparing the at least one audio or video component to a range of known values.
11. The system according to claim 8 , further comprising an entropy value generator configured to derive entropy values for each generated histogram, and wherein the audio visual data classifier is configured to generate the control metadata that represents a type of the audio visual content based on the derived entropy values.
12. The system according to claim 8 , wherein the generated control metadata comprises one or more of a mean, a variance, a skew and a kurtosis of the frequency of occurrences.
13. The system according to claim 8 , wherein the audio visual content comprises a video stream of video frames, with the at least one fingerprint generated for every frame in the video stream.
14. The system according to claim 8 , wherein the audio visual data classifier is further configured to generate the control metadata as a weighted sum of outputs from a plurality of different comparison functions, wherein respective weights and functions are previously selected by a training process where candidate sets of comparison functions are applied iteratively to sets of statistical data and entropy data derived from an analysis of fingerprint data from known types of audio visual content.
15. The system according to claim 8 , wherein the test signal detector comprises a shot-change detector configured to identify isolated high values of the at least one content fingerprint by comparing respective value differences between a first fingerprint and preceding and succeeding content fingerprints with a threshold, and the audio visual data classifier is configured to generate shot-change metadata when the respective value differences are above a predetermined threshold.
16. A system for managing audio visual content, the system comprising:
a separator configured to separate out at least one audio or video component from at least one content fingerprint in a received audio visual fingerprint stream, with the at least one audio or video component corresponding to a content characteristic of an audio visual stream associated with the received audio visual fingerprint stream;
a test signal detector configured to generate at least one fingerprint value, respectively, of the separated at least one audio or video component by executing a statistical analysis operation of the separated at least one audio or video component; and
an audio visual data classifier configured to generate control metadata based on the generated at least one fingerprint value;
wherein the at least one audio or video component is an audio sample component, and the audio visual data classifier comprises an audio level detector configured to generate audio level metadata indicating a loudness level of the audio visual stream when a sustained number of occurrences of the audio sample component within the fingerprint sequence of the audio visual stream are below a low-level threshold;
wherein the generated control metadata configures an automatic control system for distributing the audio visual stream.
17. The system according to claim 16 , wherein the audio visual data classifier is further configured to generate the control metadata by comparing a frequency of occurrence values of the generated at least one fingerprint value to at least one known type of audio visual content.
18. The system according to claim 17 , further comprising a histogram generator configured to generate a histogram based on the frequency of occurrence values of the generated at least one fingerprint value within the fingerprint sequence of the audio visual fingerprint stream.
19. The system according to claim 18 , further comprising an entropy value generator configured to derive entropy values for each generated histogram, and wherein the audio visual data classifier is configured to generate control metadata for the content controller that represents a type of the audio visual content based on the derived entropy values.
20. The system according to claim 19 , wherein the generated control metadata comprises one or more of a mean, a variance, a skew and a kurtosis of the frequency of occurrences.
21. The system according to claim 16 , wherein the audio visual content comprises a video stream of video frames, with the at least one fingerprint generated for every frame in the video stream.
22. The system according to claim 16 , wherein the audio visual data classifier is further configured to generate a control metadata as a weighted sum of outputs from a plurality of different comparison functions, wherein respective weights and functions are previously selected by a training process where candidate sets of comparison functions are applied iteratively to sets of statistical data and entropy data derived from an analysis of fingerprint data from known types of audio visual content.
23. The system according to claim 16 , wherein the audio visual data classifier comprises a shot-change detector configured to identify isolated high values of the at least one content fingerprint by comparing respective value differences between a first fingerprint and preceding and succeeding content fingerprints with a threshold, and the audio visual data classifier is configured to generate shot-change metadata when the respective value differences are above a predetermined threshold.
24. The system according to claim 16 , wherein the statistical analysis operation comprises at least one of comparing the at least one audio or video component to a predetermined threshold and comparing the at least one audio or video component to a range of known values.