Method and device for generating and detecting fingerprints for synchronizing audio and video
View Patent ↗This invention relates to a device and a method of generating a first and a second fingerprint ( 102,104 ) usable for synchronisation of at least two signals ( 101,103 ) and corresponding method and device for synchronising two or more signals. A fingerprint pair is generated on the basis of a segment of a first signal e.g. an audio signal and of a segment of a second signal e.g. a video signal at each synchronisation time point. The generated fingerprint pair(s) are stored in a database ( 203 ) and communicated or distributed to a synchronisation device ( 303 ). During synchronisation, fingerprint(s) of the audio signal and fingerprint(s) of the video signal to be synchronised are generated and matched against fingerprints in the database. When a match is found, the fingerprints also determine the synchronisation time point, which is used to synchronise the two signals. In this way, a simple, reliable and efficient way of synchronising at least two signals is obtained. Further, this is enabled without modifying either the first and second signal (or subsequent signals). The signals may even be distorted or changed to some extent while still enabling synchronisation.
1. A method comprising:
accessing a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from a first signal and a second fingerprint of the fingerprint pair is generated from a second signal,
the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other; and
using a processor, processing a search for the fingerprint pair by a device that is configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal.
2. The method of claim 1 further comprising:
the processing of the search for the fingerprint pair includes enabling access to a database that is configured to provide the fingerprint pair to the device in response to the search.
3. The method of claim 1 further comprising:
the processing of the search for the fingerprint pair includes providing the fingerprint pair to the device.
4. The method of claim 1 , wherein:
the time point at which the first and second signals are synchronized to each other corresponds to a location within the first signal; and
the method further comprises storing the fingerprint pair with a representation of an offset between the location within the first signal and a reference time.
5. The method of claim 4 , wherein:
the time point at which the first and second signals are synchronized to each other corresponds to a location within the second signal; and
the method further comprises storing the fingerprint pair with a representation of a further offset between the location within the second signal and the reference time.
6. The method of claim 1 , wherein:
the first signal is an audio signal; and
the second signal is a video signal.
7. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
accessing a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from a first signal and a second fingerprint of the fingerprint pair is generated from a second signal,
the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other; and
processing a search for the fingerprint pair by a device that is configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal.
8. The non-transitory machine-readable storage medium of claim 7 , wherein:
the time point at which the first and second signals are synchronized to each other corresponds to a location within the first signal; and
the operations further comprise storing the fingerprint pair with a representation of an offset between the location within the first signal and a reference time.
9. A system comprising:
a microprocessor configured by a fingerprint generator to access a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from a first signal and a second fingerprint of the fingerprint pair is generated from a second signal,
the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other; and
a database configured to process a search for the fingerprint pair by a device that is configured to find a first match to the first fingerprint among first fingerprints generated from the first signal and find a second match to the second fingerprint among second fingerprints generated from the second signal.
10. The system of claim 9 , wherein:
the database is configured to process the search for the fingerprint pair by providing the fingerprint pair to the device in response to the search.
11. A method comprising:
generating first fingerprints from a first signal and second fingerprints from a second signal out of synchronization with the first signal;
accessing a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from the first signal and a second fingerprint of the fingerprint pair is generated from the second signal,
the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other;
finding a first match to the first fingerprint among the first fingerprints and finding a second match to the second fingerprint among the second fingerprints; and
using a processor, synchronizing the first and second signals based on the first and second matches and based on the time point indicated by the fingerprint pair.
12. The method of claim 11 , wherein:
the accessing of the fingerprint pair includes receiving the fingerprint pair from a database that stores the fingerprint pair.
13. The method of claim 11 , wherein:
the accessing of the fingerprint pair includes receiving the fingerprint pair from a server that generated the fingerprint pair.
14. The method of claim 11 , wherein:
the synchronizing of the first and second signals includes shifting the second signal relative to the first signal by a difference between a location of the first match within the first signal and a location of the second match within the second signal.
15. The method of claim 11 , wherein:
the synchronizing of the first and second signals includes shifting an out-pointer of a buffer that buffers data of the first signal within a synchronization device.
16. The method of claim 15 , wherein:
the time point at which the first and second signals are synchronized to each other corresponds to a location within the first signal;
the method further comprises accessing a representation of an offset between the location within the first signal and a reference time; and
the shifting of the out-pointer of the buffer is based on representation of the offset.
17. The method of claim 16 , wherein:
the time point at which the first and second signals are synchronized to each other corresponds to a location within the second signal;
the method further comprises accessing a representation of a further offset between the location within the second signal and the reference time; and
the synchronizing of the first and second signals includes shifting a further out-pointer of a buffer that buffers data of the second signal within the synchronization device,
the shifting of the further out-pointer being based on the representation of the further offset.
18. The method of claim 11 , wherein:
the first signal is in audio signal; and
the second signal is a video signal.
19. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
generating first fingerprints from a first signal and second fingerprints from a second signal out of synchronization with the first signal;
accessing a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from the first signal and a second fingerprint of the fingerprint pair is generated from the second signal,
the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other;
finding a first match to the first fingerprint among the first fingerprints and finding a second match to the second fingerprint among the second fingerprints; and
synchronizing the first and second signals based on the first and second matches and based on the time point indicated by the fingerprint pair.
20. The non-transitory machine-readable storage medium of claim 19 , wherein:
the synchronizing of the first and second signals includes shifting the second signal relative to the first signal by a difference between a location of the first match within the first signal and a location of the second match within the second signal.
21. A device comprising:
a fingerprint detector configured to generate first fingerprints from a first signal and second fingerprints from a second signal out of synchronization with the first signal; and
a microprocessor configured by synchronization circuit to:
access a fingerprint pair of which a first fingerprint of the fingerprint pair is generated from the first signal and a second fingerprint of the fingerprint pair is generated from the second signal, the fingerprint pair indicating a time point at which the first and second signals are synchronized to each other;
find a first match to the first fingerprint among the first fingerprints and finding a second match to the second fingerprint among the second fingerprints; and
synchronize the first and second signals based on the first and second matches and based on the time point indicated by the fingerprint pair.
22. The device of claim 21 , wherein:
the synchronization circuit configures the microprocessor to synchronize the first and second signals by shifting an out-pointer of a buffer that buffers data of the first signal.