Granted Patent B2

US 7,461,002 · App. 10/478,398 · Granted Dec 2, 2008

Assignee: Dolby Laboratories Licensing Corporation

Abstract

Claims (36)

1. A method for time aligning first and second audio signals, wherein one signal has been derived from the other or both have been derived from another signal, comprising

deriving reduced-information characterizations of said audio signals, the characterizations being composed of less information than the audio signals themselves, wherein said reduced-information characterizations represent at least the boundaries of auditory events resulting from the division of each of said audio signals into auditory events, each of which auditory events tends to be perceived as separate and distinct, wherein each audio signal is divided into auditory events by

detecting changes in signal characteristics with respect to time in the audio signal, and

identifying a continuous succession of auditory event boundaries in the audio signal, in which every change in signal characteristics with respect to time exceeding a threshold defines a boundary, wherein each auditory event is an audio segment between adjacent boundaries and there is only one auditory event between such adjacent boundaries, each boundary representing the end of the preceding event and the beginning of the next event such that a continuous succession of auditory events is obtained, wherein neither auditory event boundaries, auditory events, nor any characteristics of an auditory event are known in advance of identifying the continuous succession of auditory event boundaries and obtaining the continuous succession of auditory events,

the characterizations having substantially the same relative timing relationship as the audio signals from which they are derived,

calculating the time offset of one characterization with respect to the other characterization, and

modifying the temporal relationship of said audio signals with respect to each other in response to said time offset such that said audio signals are more closely aligned in time.

2. The method of claim 1 wherein each of said audio signals is accompanied by a respective other signal and wherein each of said reduced-information characterizations, prior to said calculating and modifying, is embedded into the respective other signal that is carried with the audio signal from which the respective reduced-information characterization was derived.

3. The method of claim 2 wherein said other signals are video signals.

4. A method for time aligning an audio signal and another signal comprising

deriving a reduced-information characterization of the audio signal and embedding said characterization in the other signal when the audio signal and other signal are substantially in synchronism, wherein said characterization represents at least the boundaries of auditory events resulting from the division or said audio signal into auditory events, each of which auditory events tends to be perceived as separate and distinct, wherein the audio signal is divided into auditory events by

detecting changes in signal characteristics with respect to time in the audio signal, and

identifying a continuous succession of auditory event boundaries in the audio signal, in which every change in signal characteristics with respect to time exceeding a threshold defines a boundary, wherein each auditory event is an audio segment between adjacent boundaries and there is only one auditory event between such adjacent boundaries, each boundary representing the end of the preceding event and the beginning of the next event such that a continuous succession of auditory events is obtained, wherein neither auditory event boundaries, auditory events, nor any characteristics of an auditory event are known in advance of identifying the continuous succession of auditory event boundaries and obtaining the continuous succession of auditory events,

recovering the embedded characterization of said audio signal from said other signal after said audio signal and said other signal have been subjected to differential time offsets and deriving a reduced-information characterization of said audio signal from said audio signal in the same way the embedded characterization of the audio signal was derived based on auditory scene analysis,

calculating the time offset of one characterization with respect to the other characterization,

modifying the temporal relationship of the audio signal with respect to the other signal in response to said time offset such that the audio signal and other signal are more closely aligned in time.

5. The method of claim 4 wherein said other signal is a video signal.

6. The method of claim 1 wherein calculating a time offset includes performing a cross-correlation of said characterizations.

7. The method of claim 4 wherein calculating a time offset includes performing a cross-correlation of said characterization.

8. The method of any one of claims 1 - 6 and 7 wherein said reduced-information characterizations also represent the dominant frequency subband of each of said auditory events.

9. A method for time aligning an audio signal and one other signal, which signals are subjected to differential time offsets during storage or transmission comprising

before said audio signal and said other signal have been subjected to differential time offsets, deriving a first reduced-information characterization of the audio signal, and embedding said first characterization in the other signal, wherein said first characterization represents at least the boundaries of auditory events resulting from the division of said audio signal into auditory events, each of which auditory events tends to be perceived as separate and distinct, wherein the audio signal is divided into auditory events by

detecting changes in signal characteristics with respect to time in the audio signal, and

identifying a continuous succession of auditory event boundaries in the audio signal, in which every change in signal characteristics with respect to time exceeding a threshold defines a boundary, wherein each auditory event is an audio segment between adjacent boundaries and there is only one auditory event between such adjacent boundaries, each boundary representing the end of the preceding event and the beginning of the next event such that a continuous succession of auditory events is obtained, wherein neither auditory event boundaries, auditory events, nor any characteristics of an auditory event are known in advance of identifying the continuous succession of auditory event boundaries and obtaining the continuous succession of auditory events,

after said audio signal and said other signal have been subjected to differential time offsets, recovering the embedded first characterization of said audio signal from said other signal and recovering a second reduced-information characterization from said audio signal in the same way the first characterization of the audio signal was derived based on auditory scene analysis, the first and second characterizations each being composed of less information than the audio signal from which each is derived,

calculating the tune offset of the first characterization with respect to the second characterization, and

modifying the temporal relationship of the audio signal with respect to the other signal in response to said time offset such that the audio signal and other signal are more closely aligned in time.

10. A method for time aligning an audio signal and another signal, which signals have been subjected to differential time offsets during storage or transmission after a first reduced-information characterization had been derived from the audio signal and embedded in the other signal, said characterization representing at least the boundaries of auditory events resulting from the division of said audio signal into auditory events, each of which auditory events tends to be perceived as separate and distinct, wherein the audio signal is divided into auditory events by

detecting changes in signal characteristics with respect to time in the audio signal, and

identifying a continuous succession of auditory event boundaries in the audio signal, in which every change in signal characteristics with respect to time exceeding a threshold defines a boundary, wherein each auditory event is an audio segment between adjacent boundaries and there is only one auditory event between such adjacent boundaries, each boundary representing the end of the preceding event and the beginning of the next event such that a continuous succession of auditory events is obtained, wherein neither auditory event boundaries, auditory events, nor any characteristics of an auditory event are known in advance of identifying the continuous succession of auditory event boundaries and obtaining the continuous succession of auditory events, comprising

recovering the embedded first characterization of said audio signal from said other signal and deriving a second reduced-information characterization from said audio signal in the same way the first characterization of the audio signal was derived based on auditory scene analysis, the first and second characterizations each being composed of less information than the audio signal from which each is derived,

calculating die time offset of to first characterization with respect to the second characterization, and

modifying the temporal relationship of the audio signal with respect to the other signal in response to said time offset such that the audio signal and other signal are more closely aligned in time.

11. The method of claim 9 or 10 wherein said other signal is a video signal.

12. The method of claim 9 or 10 , wherein calculating a time offset includes performing a cross-correlation of said characterizations.

13. The method of claim 9 or claim 10 wherein said reduced-information characterizations also represent the dominant frequency subband of each of said auditory events.

Assignments (1)

Continuity (8)