IP Library › Granted Patent US 10,158,927
Granted Patent B1
US 10,158,927 · App. 13/603,571 · Granted Dec 18, 2018

Systems and methods for detecting audio-video synchronization using timestamps

Inventors: Yong Lei (Cupertino, CA); Ke Yang (Cupertino, CA)
Assignee: GOOGLE LLC
H04N21/8547H04N21/2368H04N21/4307
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,158,927
App. No.
13/603,571
Filed
Sep 5, 2012
Granted
Dec 18, 2018
Kind
B1
Examiner
MA, WING
Art Unit
2456
USPC
709/231
Abstract

Systems and techniques for testing audio-video synchronization using one or more timestamps are presented. On a media sender end, a media stream with a video stream and an audio stream is generated. A matrix barcode timestamp is added onto a video frame of the video stream at a particular time. Furthermore, a tone timestamp is embedded into the audio stream at the particular time. Additionally, on a media receiver end, a modified media stream is received. For example, the video stream with the matrix barcode timestamp at the particular time and the audio stream with the tone timestamp at the particular time are received. The matrix barcode timestamp is decoded to determine a first time value and the tone timestamp is decoded to determine a second time value. The first time value is compared with the second time value to determine synchronization of the video stream and the audio stream.

Claims (35)

1. A system, comprising:

a processor; and

a memory communicatively coupled to the processor, the memory having stored therein computer-executable instructions, comprising:

a media component configured to:

receive a matrix barcode timestamp that is part a video stream of a media stream that is being received, wherein the matrix barcode timestamp has encoded a first time value in a machine readable visual format, and

receive a tone timestamp that is part of an audio stream of the media stream that is being received, wherein the tone timestamp encodes a second time value in a machine hearable audio format;

a parse component configured to determine a first wall clock time value associated with a receipt of the matrix barcode timestamp and a second wall clock time value associated with a receipt of the tone timestamp;

a comparison component configured to:

compare the first wall clock time value with the second wall clock time value;

determine a difference between the first wall clock time value and the second wall clock time value;

determine whether a synchronization error exists between of the video stream and the audio stream using the difference between the first wall clock time value and the second wall clock time value; and

generate a synchronization signal responsive to determining a synchronization error exists between the video stream and the audio stream; and

a report component configured to upload synchronization results to a server, wherein the tone timestamp is a dual-tone multifrequency (DTMF) tone timestamp.

2. The system of claim 1 , wherein the parse component is further configured to generate a screenshot of the video stream in order to decode the matrix barcode timestamp.

3. The system of claim 1 , wherein the media component is further configured to receive the media stream at one or more network speeds.

4. The system of claim 1 , wherein the parse component is further configured to:

decode the matrix barcode timestamp to determine the first time value, and decode the tone timestamp to determine the second time value, and wherein the

comparison component is further configured to:

compare the first wall clock time value with the first time value to determine latency of the video stream, and

compare the second wall clock time value with the second time value to determine latency of the audio stream.

5. The system of claim 1 , wherein the media stream is a looped video clip.

6. A method, comprising:

receiving, by a system including a processor, a matrix barcode timestamp that is part a video stream of a media stream that is being received, wherein the matrix barcode timestamp has encoded a first time value in a machine readable visual format;

receiving, by the system, a tone timestamp that is part of an audio stream of the media stream that is being received, wherein the tone timestamp encodes a second time value in a machine hearable audio format video, wherein the tone timestamp is a dual-tone multifrequency (DTMF) tone timestamp;

determining, by the system, a first wall clock time value associated with a receipt of the matrix barcode timestamp;

determining, by the system, a second wall clock time value associated with a receipt of the tone timestamp;

comparing, by the system, the first wall clock time value with the second wall clock time value;

determining whether a synchronization error exists between the video stream and the audio stream using a difference between the first wall clock value and the second wall clock value;

generating a synchronization-signal responsive to determining a synchronization error exists between the video stream and the audio stream; and

uploading synchronization results to a server.

7. The method of claim 6 , further comprising:

decoding by the system, the matrix barcode timestamp to determine the first time value; comparing, by the system, the first wall clock time value with the first time value to determine latency of the video stream, decoding, by the system, the tone timestamp to determine the second time value: and

comparing, by the system, and the second wall clock time value with the second time value to determine latency of the audio stream.

8. The method of claim 6 , further comprising generating a screenshot of the video stream in order to decode the matrix barcode timestamp.

9. The method of claim 6 , further comprising receiving the media stream at one or more network speeds.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2012
From: LEI, YONG; YANG, KE
To: GOOGLE INC.
Reel/Frame 028897/0398 →
Cited By (5)
US 12,225,259 US 12,300,257 US 12,393,190 US 12,432,410 US 12,464,180