Granted Patent B2

US 8,860,883 · App. 12/627,728 · Granted Oct 14, 2014

Abstract

Claims (82)

1. A method for generating a video signature representative of a content of a video signal, the method comprising:

receiving pixel data of each of a plurality of given pixels in a first image of the video signal,

receiving pixel data of each of the plurality of given pixels in a second image subsequent to the first image,

for each given pixel of the plurality of given pixels,

comparing corresponding pixel data of the given pixel of the first image with the corresponding pixel data of the given pixel in the second image to provide a corresponding indication of a difference between the pixel data of the given pixel of the first image and the pixel data of the given pixel in the second image;

if the difference is greater than a threshold, incrementing a counter;

normalizing the indication of the counter by dividing the counter signal by a value proportional to N, wherein N is the number of given pixels; and

providing the normalized indication of the counter to thereby generate said video signature.

2. The method as claimed in claim 1 , further comprising selecting a plurality of given pixels according to at least one criteria.

3. The method as claimed in claim 2 , wherein the at least one criteria comprises a spatial criteria.

4. The method as claimed in claim 1 , wherein the pixel data of the given pixel in the first image and the pixel data of the given pixel in the second image each comprise an 8-bit numerical value between zero (0) and two hundred and fifty five (255), and wherein the threshold comprises a difference between the numerical values of the pixel data of the given pixel of the first image and the pixel data of the given pixel of the second image that is equal to a numerical value of thirty two (32).

5. A method for generating an audio signature representative of a content of an audio signal, the method comprising:

receiving an audio signal;

performing a first filtering of the audio signal to provide a first filtered audio signal;

performing a second filtering of the audio signal to provide a second filtered audio signal;

comparing the first filtered audio signal to the second filtered audio signal; and

assigning a value depending on the comparison, and decimating a plurality of assigned values by removing N−1 of every N assigned values, wherein N is a predetermined number of values, to thereby generate the audio signature representative of the content of the audio signal.

6. The method as claimed in claim 5 , wherein said first filtering comprises detecting an envelope signal of the received audio signal.

7. The method as claimed in claim 6 , wherein said second filtering comprises computing an average value of the received audio signal.

8. An audio signature extraction unit for providing an audio signature signal AS i representative of a content of an audio signal A i , the audio signature extraction unit comprising:

a first filtering unit for receiving the audio signal A i and for providing a first filtered signal;

a second filtering unit for receiving the audio signal A i and providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the audio signature signal AS i representative of a content of an audio signal; and

a decimator for receiving the audio signature signal AS i and for decimating the audio signature signal AS i by removing N−1 of every N assigned values, wherein N is a predetermined number of values.

9. The audio signature extraction unit as claimed in claim 8 , wherein said first filtering unit comprises an envelope detector for providing a signal E s , indicative of an envelope of the audio signal A i .

10. The audio signature extraction unit as claimed in claim 9 , wherein said second filtering unit comprises a mean detector for providing a signal M s , indicative of a mean the audio signal A i .

11. The audio signature extraction unit as claimed in claim 10 , further comprising an absolute value providing unit for receiving the audio signal A i and for providing a signal indicative of an absolute value of the value of the audio signal A i ; wherein each of said mean detector and said envelope detector receives said signal indicative of an absolute value of the value of the audio signal A i .

12. A video signature extraction unit for providing a video signature signal VS i representative of a content of an incoming image signal V i the video signature extraction unit comprising:

a first image pixel data providing unit for receiving the incoming image signal V i and for providing a corresponding pixel data P k of a pixel k of a plurality of given pixels 1 to N of a first image of the incoming image signal V i ;

a second image pixel data providing unit for receiving the incoming image signal Vi and for providing a corresponding pixel data C k of the corresponding given pixel k in a second image of the incoming image signal V i ;

a comparator for receiving the corresponding pixel data P k and the corresponding pixel data C k and for comparing the corresponding pixel data P k with the corresponding pixel data C k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal, normalizing the counter signal by dividing the counter signal by a value proportional to N, wherein N is the number of given pixels, and for providing the video signature signal VS i .

13. The video signature extraction unit as claimed in claim 12 , wherein P k and C k each comprise an 8-bit numerical value between zero (0) and two hundred and fifty five (255), and wherein the result signal is a logic value one (1) if |P k −C k | is greater than a numerical value of thirty two (32) and a logic value zero (0) otherwise.

14. The video signature extraction unit as claimed in claim 12 , further comprising a windowing unit for receiving the incoming image signal V i and for providing a selected signal W i .

15. An apparatus for providing an indication of a lip sync, the apparatus comprising:

a first signature extraction unit for receiving a first audio signal and a first video signal and for providing a first video signature signal and a first audio signature signal;

a second signature extraction unit for receiving a second audio signal and a second video signal and for providing a second video signature signal and a second audio signature signal;

a signature analysis unit for receiving the first video signature signal, the first audio signature signal, the second video signature signal and the second audio signature signal and for correlating the first video signature signal with the second video signature signal to provide a video delay and for further correlating the first audio signature signal with the second audio signature signal to provide an audio delay, said signature analysis unit for further comparing said audio delay with said video delay and for providing an indication of a lip sync is said audio delay is different from said video delay,

wherein correlating the first video signature signal with the second video signature to provide a video delay comprises performing a convolution of the first video signature signal and second video signature signal to generate a first convolution signal, and determining a video matching factor inversely proportional to a minimum value of the first convolution signal divided by a sum of one of the first and second video signature signals, wherein the first video signature signal and second video signature signal are correlated if the value of the video matching factor is above a first predetermined threshold, and

wherein correlating the first audio signature signal with the second audio signature to provide an audio delay comprises performing a convolution of the first audio signature signal and second audio signature signal to generate a second convolution signal, and determining an audio matching factor inversely proportional to a minimum value of the second convolution signal divided by a size of one of the first and second audio signature signals, wherein the first audio signature signal and second audio signature signal are correlated if the value of the audio matching factor is above a second predetermined threshold;

wherein said first signature extraction unit comprises a first audio signature extraction unit for receiving said first audio signal and for providing said first audio signature signal, further wherein said first signature extraction unit comprises a first video signature extraction unit for receiving said first video signal and for providing said first video signature signal further wherein said first audio signature extraction unit comprises:

a first filtering unit for receiving the first audio signal and for providing a first filtered signal;

a second filtering unit for receiving the first audio signal and for providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the first audio signature signal.

16. An apparatus for providing an indication of a lip sync, the apparatus comprising:

a first signature extraction unit for receiving a first audio signal and a first video signal and for providing a first video signature signal and a first audio signature signal;

a second signature extraction unit for receiving a second audio signal and a second video signal and for providing a second video signature signal and a second audio signature signal;

a signature analysis unit for receiving the first video signature signal, the first audio signature signal, the second video signature signal and the second audio signature signal and for correlating the first video signature signal with the second video signature signal to provide a video delay and for further correlating the first audio signature signal with the second audio signature signal to provide an audio delay, said signature analysis unit for further comparing said audio delay with said video delay and for providing an indication of a lip sync is said audio delay is different from said video delay,

wherein correlating the first video signature signal with the second video signature to provide a video delay comprises performing a convolution of the first video signature signal and second video signature signal to generate a first convolution signal, and determining a video matching factor inversely proportional to a minimum value of the first convolution signal divided by a sum of one of the first and second video signature signals, wherein the first video signature signal and second video signature signal are correlated if the value of the video matching factor is above a first predetermined threshold, and

wherein correlating the first audio signature signal with the second audio signature to provide an audio delay comprises performing a convolution of the first audio signature signal and second audio signature signal to generate a second convolution signal, and determining an audio matching factor inversely proportional to a minimum value of the second convolution signal divided by a size of one of the first and second audio signature signals, wherein the first audio signature signal and second audio signature signal are correlated if the value of the audio matching factor is above a second predetermined threshold;

wherein said first signature extraction unit comprises a first audio signature extraction unit for receiving said first audio signal and for providing said first audio signature signal, further wherein said first signature extraction unit comprises a first video signature extraction unit for receiving said first video signal and for providing said first video signature signal; and

wherein said first video signature extraction unit comprises:

a first image pixel data providing unit for receiving said first video signal and for providing a corresponding pixel data P k of a pixel k of a plurality of given pixels 1 to N of a first image of said first video signal;

a second image pixel data providing unit for receiving said first video signal and for providing a corresponding pixel data C k of the corresponding given pixel k in a second image of said first video signal;

a comparator for receiving the corresponding pixel data P k and the corresponding pixel data C k and for comparing the corresponding pixel data P k with the corresponding pixel data C k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal and for providing the first video signature signal.

17. An apparatus for providing an indication of a lip sync, the apparatus comprising:

a first signature extraction unit for receiving a first audio signal and a first video signal and for providing a first video signature signal and a first audio signature signal;

a second signature extraction unit for receiving a second audio signal and a second video signal and for providing a second video signature signal and a second audio signature signal;

a signature analysis unit for receiving the first video signature signal, the first audio signature signal, the second video signature signal and the second audio signature signal and for correlating the first video signature signal with the second video signature signal to provide a video delay and for further correlating the first audio signature signal with the second audio signature signal to provide an audio delay, said signature analysis unit for further comparing said audio delay with said video delay and for providing an indication of a lip sync is said audio delay is different from said video delay,

wherein correlating the first video signature signal with the second video signature to provide a video delay comprises performing a convolution of the first video signature signal and second video signature signal to generate a first convolution signal, and determining a video matching factor inversely proportional to a minimum value of the first convolution signal divided by a sum of one of the first and second video signature signals, wherein the first video signature signal and second video signature signal are correlated if the value of the video matching factor is above a first predetermined threshold, and

wherein correlating the first audio signature signal with the second audio signature to provide an audio delay comprises performing a convolution of the first audio signature signal and second audio signature signal to generate a second convolution signal, and determining an audio matching factor inversely proportional to a minimum value of the second convolution signal divided by a size of one of the first and second audio signature signals, wherein the first audio signature signal and second audio signature signal are correlated if the value of the audio matching factor is above a second predetermined threshold;

wherein said second signature extraction unit comprises a second audio signature extraction unit for receiving said second audio signal and for providing said second audio signature signal, further wherein said second signature extraction unit comprises a second video signature unit for receiving said second video signal and for providing said second video signature signal;

wherein said second audio signature extraction unit comprises:

a first filtering unit for receiving the second audio signal and for providing a first filtered signal;

a second filtering unit for receiving the second audio signal and for providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the second audio signature signal.

18. An apparatus for providing an indication of a lip sync, the apparatus comprising:

a first signature extraction unit for receiving a first signal and first video signal and for providing a first video signature signal and a first audio signature signal;

a second signature extraction unit for receiving a second audio signal and a second video signal and for providing a second video signature signal and a second audio signature signal;

a signature analysis unit for receiving the first video signature signal, the first audio signature signal, the second video signature signal and the second audio signature signal and for correlating the first video signature signal with the second video signature signal to provide a video delay and for further correlating the first audio signature signal with the second audio signature signal to provide an audio delay, said signature analysis unit for further comparing said audio delay with said video delay and for providing an indication of a lip sync is said audio delay is different from said video delay,

wherein correlating the first video signature signal with the second video signature to provide a video delay comprises performing a convolution of the first video signature signal and second video signature signal to generate a first convolution signal, and determining a video matching factor inversely proportional to a minimum value of the first convolution signal divided by a sum of one of the first and second video signature signals, wherein the first video signature signal and second video signature signal are correlated if the value of the video matching factor is above a first predetermined threshold, and

wherein correlating the first audio signature signal with the second audio signature to provide an audio delay comprises performing a convolution of the first audio signature signal and second audio signature signal to generate a second convolution signal, and determining an audio matching factor inversely proportional to a minimum value of the second convolution signal divided by a size of one of the first and second audio signature signals, wherein the first audio signature signal and second audio signature signal are correlated if the value of the audio matching factor is above a second predetermined threshold;

wherein said second signature extraction unit comprises a second audio signature extraction unit for receiving said second audio signal and for providing said second audio signature signal, further wherein said second signature extraction unit comprises a second video signature unit for receiving said second video signal and for providing said second video signature signal;

wherein said second video signature extraction unit comprises:

a first image pixel data providing unit for receiving said second video signal and for providing a corresponding pixel data P k of a pixel k of a plurality of given pixels 1 to N of a first image of said second video signal;

a second image pixel data providing unit for receiving said second video signal and for providing a corresponding pixel data C k of the corresponding given pixel k in a second image of said second video signal;

a comparator for receiving the corresponding pixel data P k and the corresponding pixel data C k and for comparing the corresponding pixel data P k with the corresponding pixel data C k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal and for providing the second video signature signal.

Assignments (8)