Method, apparatus and system of synchronizing audio with video

1. A method of synchronizing audio with video, comprising:

receiving, by a receiving device, a video signal embedded with a video sync signal and an audio signal embedded with an audio sync signal provided by a sending device, the video sync signal is embedded in the video signal by setting a value of a pixel at a pre-determined position in a video frame of the video signal to be a value corresponding to a value of the video sync signal or setting a pixel with a pre-determined value to be at a position corresponding to a value of the video sync signal in a video frame of the video signal;

extracting the video sync signal from the video signal by mapping one of the following to a value of the video sync signal:

a value of a pixel at the pre-determined position of a video frame, or

a relation between positions of at least two pixels each having a pre-determined value in a video frame, or

a relation between positions of a pixel having a pre-determined value in at least two video frames;

extracting the audio sync signal from the audio signal;

obtaining a time difference by comparing the video sync signal with the audio sync signal; and

processing the video signal and the audio signal according to the time difference to have the processed audio signal synchronized with the processed video signal during playback.

2. The method of claim 1 , wherein processing the video signal and the audio signal according to the time difference comprises:

advancing or delaying one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

3. The method of claim 1 , wherein processing the video signal and the audio signal according to the time difference comprises:

providing the time different to the sending device which advances or delays one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

4. The method of claim 1 , wherein

the video sync signal is embedded into at least one video frames; and

the audio sync signal is embedded into an audio segment which is collected at the same time with the video frames or has the same playback time position with the video frames in a video file from which the video signal and the audio signal are extracted.

5. The method of claim 1 , wherein extracting the video sync signal from the video signal comprises at least one of:

obtaining a value of a pixel at a pre-determined position in a video frame;

obtaining a relation between positions of at least two pixels each having a pre-determined value in a video frame; or

obtaining a relation between positions of a pixel having a pre-determined value in two video frames.

6. The method of claim 1 , wherein extracting the audio sync signal from the audio signal comprises at least one of:

mapping an amplitude or a frequency of a sine wave at a frequency band insensitive to human ears of the audio signal to a value of the audio sync signal;

mapping an amplitude or a frequency of a rectangular wave at a frequency band insensitive to human ears of the audio signal to a value of the audio sync signal;

mapping an amplitude or a frequency of a triangular wave at a frequency band insensitive to human ears of the audio signal to a value of the audio sync signal; or

mapping an amplitude or a frequency of a pulse signal at a frequency band insensitive to human ears of the audio signal to a value of the audio sync signal.

7. The method of claim 1 , further comprising at least one of:

embedding watermark information specifying copyright into the video sync signal;

embedding watermark information specifying copyright into the audio sync signal;

embedding video control information for controlling video playback into the video sync signal;

embedding auxiliary information related with video playback into the video sync signal;

embedding audio control information for controlling audio playback into the audio sync signal; or

embedding auxiliary information related with audio playback into the audio sync signal.

8. The method of claim 1 , wherein extracting the video sync signal from the video signal comprises:

obtaining a first sequence of values of the video sync signal, each of the values of the video sync signal is obtained from a different set of video frames of the video signal; and

obtaining a second sequence of values of the audio sync signal, each of the values of the audio sync signal is obtained from a different audio segment of the audio signal;

wherein obtaining the time difference by comparing the video sync signal with the audio sync signal comprises: obtaining the time difference by comparing the first sequence with the second sequence.

9. The method of claim 8 , wherein obtaining the first sequence of the values of the video sync signal comprises:

applying a sliding window having a pre-determined length to values of the video sync signal obtained from a plurality sets of video frames of the video signal; and

determining values within the sliding window to be the first sequence.

10. The method of claim 8 , wherein obtaining the second sequence of the values of the audio sync signal comprises:

obtaining values of the audio sync signal from a plurality of audio signal segments of the audio signal;

applying a sliding window having a pre-determined length to the values of the audio sync signal; and

determining values within the sliding window to be the second sequence.

11. The method of claim 1 , wherein mapping the value of the pixel at the pre-determined position of the video frame to the value of the video sync signal comprises:

obtaining a pre-determined value range to which the value of the pixel belongs; and

determining a pre-defined value associated with the value range to be the value of the video sync signal.

12. An apparatus of synchronizing audio with video, comprising a processor and a memory storing computer-executable instructions that, when executed by the processor, cause the processor to:

receive a video signal embedded with a video sync signal and an audio signal embedded with an audio sync signal provided by a sending device, the video sync signal is embedded in the video signal by setting a value of a pixel at a pre-determined position in a video frame of the video signal to be a value corresponding to a value of the video sync signal or by setting a pixel with a pre-determined value to be at a position corresponding to a value of the video sync signal in a video frame of the video signal;

extract the video sync signal from the video signal by mapping one of the following to a value of the video sync signal:

a value of a pixel at the pre-determined position of a video frame, or

a relation between positions of at least two pixels each having a pre-determined value in a video frame, or

a relation between positions of a pixel having a pre-determined value in at least two video frames;

extract the audio sync signal from the audio signal;

obtain a time difference by comparing the video sync signal with the audio sync signal; and

process the video signal and the audio signal according to the time difference.

13. The apparatus of claim 12 , wherein the instructions further cause the processor to:

advance or delay one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

14. The apparatus of claim 12 , wherein the instructions further cause the processor to:

provide the time different to the sending device which advances or delays one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

15. The apparatus of claim 12 , wherein the instructions further cause the processor to:

obtain a first sequence of values of the video sync signal, each of the values of the video sync signal is obtained from a different set of video frames of the video signal;

obtain a second sequence of values of the audio sync signal, each of the values of the audio sync signal is obtained from a different audio segment of the audio signal; and

obtain the time difference by comparing the first sequence with the second sequence.

16. The apparatus of claim 12 , wherein the instructions further cause the processor to:

apply a sliding window having a pre-determined length to values of the video sync signal obtained from a plurality sets of video frames of the video signal; and

determining values within the sliding window to be the first sequence.

17. A non-transitory computer-readable storage medium, storing instructions executable by at least one processor to:

receive a video signal embedded with a video sync signal and an audio signal embedded with an audio sync signal provided by a sending device, the video sync signal is embedded in the video signal by setting a value of a pixel at a pre-determined position in a video frame of the video signal to be a value corresponding to a value of the video sync signal or by setting a pixel with a pre-determined value to be at a position corresponding to a value of the video sync signal in a video frame of the video signal;

extract the video sync signal from the video signal by mapping one of the following to a value of the video sync signal:

a value of a pixel at the pre-determined position of a video frame, or

a relation between positions of at least two pixels each having a pre-determined value in a video frame, or

a relation between positions of a pixel having a pre-determined value in at least two video frames;

extract the audio sync signal from the audio signal;

obtain a time difference by comparing the video sync signal with the audio sync signal;

and

process the video signal and the audio signal according to the time difference.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions are executable by the processor to:

advance or delay one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions are executable by at least one processor to:

provide the time difference to the sending device which advances or delays one of the video signal and the audio signal to adjust the time difference between the video signal and the audio signal to make the video signal and the audio signal synchronous during playback.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions are executable by at least one processor to:

obtain a first sequence of values of the video sync signal, each of the values of the video sync signal is obtained from a different set of video frames of the video signal;

obtain a second sequence of values of the audio sync signal, each of the values of the audio sync signal is obtained from a different audio segment of the audio signal; and

obtain the time difference by comparing the first sequence with the second sequence.