IP Library Granted Patent US 12,199,746
Granted Patent B2
US 12,199,746 · App. 17/969,095 · Granted Jan 14, 2025

Mechanism for precise time synchronization in a datacenter network

Inventors: Abhishek Agarwal (Santa Clara, CA); Ye Tang (Palo Alto, CA); Prashant R. Chandra (San Jose, CA); Simon Luigi Sabato (Saratoga, CA); Hema Hariharan (Cupertino, CA)
Assignee: Google LLC
H04J3/0667
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,199,746
App. No.
17/969,095
Granted
Jan 14, 2025
Kind
B2
Abstract

Aspects of the disclosure are directed to supporting time synchronization across a datacenter network with greater accuracy. The time synchronization includes both software based and hardware based time synchronization mechanisms to provide more precise time synchronization across various nodes in the datacenter network. The software based mechanism can provide the initial coarse time synchronization while the hardware based mechanism can provide the subsequent finer time synchronization.

Claims (58)

1. A method for synchronizing time between a master hardware clock and one or more client hardware clocks, the method comprising:

receiving, by one or more processors of a client hardware clock, a time sync message;

performing, by the one or more processors, a validity check on the time sync message;

updating, by the one or more processors, a time of the client hardware clock in response to the received time sync message passing the validity check;

incrementing, by the one or more processors, a valid update count in response to the received time sync message passing the validity check; and

locking, by the one or more processors, the time of the client hardware clock to be within a threshold range from the master hardware clock in response to the valid update count incrementing above a threshold.

2. The method of claim 1 , wherein the time sync message is transmitted from one or more processors of one of a master hardware clock or an upstream client hardware clock.

3. The method of claim 1 , further comprising transmitting, with the one or more processors, the time sync message to a downstream client hardware clock.

4. The method of claim 1 , wherein updating the time of the client hardware clock further comprises at least one of:

adjusting the time a number of steps forward or backward; or

adjusting a frequency of the client hardware clock.

5. The method of claim 1 , wherein locking the client hardware clock further comprises determining the client hardware clock is within the threshold range from the master hardware clock.

6. The method of claim 1 , further comprising:

transmitting, by the one or more processors, a latency request message;

receiving, by the one or more processors, a latency response message; and

determining, by the one or more processors, a hardware latency time based on a time difference from when the latency request message was transmitted to the latency response message being received.

7. The method of claim 6 , wherein updating the time of the client hardware clock further comprises removing the hardware latency time from the time of the client hardware clock.

8. The method of claim 6 , wherein the hardware latency time is fixed.

9. The method of claim 1 , wherein performing the validity check comprises at least one of determining the time sync message is received at a port enabled for time sync message reception, determining the time sync message has a valid error check, determining the time sync message indicated a time within a threshold, or determining the time sync message has a time that is locked.

10. The method of claim 1 , further comprising skipping, by the one or more processors, an update of the time of client hardware clock in response to the received time sync message not passing the validity check.

11. The method of claim 10 , further comprising incrementing, by the one or more processors, a skip update count in response to the received time sync message not passing the validity check.

12. The method of claim 11 , further comprising resetting, by the one or more processors, the client hardware clock in response to the skip update count incrementing above a threshold.

13. A system comprising:

one or more processors; and

one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for synchronizing time between a master hardware clock and one or more client hardware clocks, the operations comprising:

receiving a time sync message;

performing a validity check on the time sync message;

updating a time of a client hardware clock in response to the received time sync message passing the validity check;

incrementing a valid update count in response to the received time sync message passing the validity check; and

locking the time of the client hardware clock to be within a threshold range from the master hardware clock in response to the valid update count incrementing above a threshold.

14. The system of claim 13 , wherein updating the time of the client hardware clock further comprises at least one of:

adjusting the time a number of steps forward or backward; or

adjusting a frequency of the client hardware clock.

15. The system of claim 13 , wherein locking the client hardware clock further comprises determining the client hardware clock is within the threshold range from the master hardware clock.

16. The system of claim 13 , wherein the operations further comprise:

transmitting a latency request message;

receiving a latency response message; and

determining a hardware latency time based on a time difference from when the latency request message was transmitted to the latency response message being received;

wherein updating the time of the client hardware clock further comprises removing the hardware latency time from the time of the client hardware clock.

17. The system of claim 13 , wherein the operations further comprise:

skipping an update of the time of client hardware clock in response to the received time sync message not passing the validity check;

incrementing a skip update count in response to the received time sync message not passing the validity check; and

resetting the client hardware clock in response to the skip update count incrementing above a threshold.

18. A non-transitory computer readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for synchronizing time between a master hardware clock and one or more client hardware clocks, the operations comprising:

receiving a time sync message;

performing a validity check on the time sync message;

updating a time of a client hardware clock in response to the received time sync message passing the validity check;

incrementing a valid update count in response to the received time sync message passing the validity check; and

locking the time of the client hardware clock to be within a threshold range from the master hardware clock in response to the valid update count incrementing above a threshold.

19. The non-transitory computer readable medium of claim 18 , wherein the operations further comprise:

transmitting a latency request message;

receiving a latency response message; and

determining a hardware latency time based on a time difference from when the latency request message was transmitted to the latency response message being received;

wherein updating the time of the client hardware clock further comprises removing the hardware latency time from the time of the client hardware clock.

20. The non-transitory computer readable medium of claim 18 , wherein the operations further comprise:

skipping an update of the time of client hardware clock in response to the received time sync message not passing the validity check;

incrementing a skip update count in response to the received time sync message not passing the validity check; and

resetting the client hardware clock in response to the skip update count incrementing above a threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2022
From: AGARWAL, ABHISHEK; TANG, YE; CHANDRA, PRASHANT R.; SABATO, SIMON LUIGI; HARIHARAN, HEMA
To: GOOGLE LLC
Reel/Frame 061477/0634 →
Continuity (2)
Related Publication 20240137140A1 · Apr 25, 2024
Related Publication 20240235710A9 · Jul 11, 2024
References Cited (27)
US 8370676B2 · Matsunaga · 2013 [cited by examiner]
US 8473638B2 · Aweya et al. · 2013 [cited by applicant]
US 8824332B2 · Gao · 2014 [cited by examiner]
US 8938636B1 · Hochschild et al. · 2015 [cited by applicant]
US 9569253B1 · Hsieh et al. · 2017 [cited by applicant]
US 9722739B2 · Chapman et al. · 2017 [cited by applicant]
US 9866339B1 · Mizrahi · 2018 [cited by applicant]
US 9882666B2 · Hsueh et al. · 2018 [cited by applicant]
US 10237008B2 · Butterworth et al. · 2019 [cited by applicant]
US 10976769B2 · Carlson et al. · 2021 [cited by applicant]
US 11153670B1 · Winzer · 2021 [cited by examiner]
US 11271838B2 · Crabtree et al. · 2022 [cited by applicant]
US 11968107B2 · Abdelhameed · 2024 [cited by examiner]
US 20100158164A1 · Oh · 2010 [cited by examiner]
US 20140355158A1 · Metzner · 2014 [cited by examiner]
US 20160330707A1 · Das et al. · 2016 [cited by applicant]
US 20180066597A1 · Fukuzumi · 2018 [cited by examiner]
US 20190018145A1 · Roovers · 2019 [cited by examiner]
US 20200285265A1 · Ranganathan · 2020 [cited by examiner]
US 20210141413A1 · Levi et al. · 2021 [cited by applicant]
US 20210185139A1 · Wang et al. · 2021 [cited by applicant]
US 20220345525A1 · Rizzo Piazza Roncoroni · 2022 [cited by examiner]
US 20240168797A1 · Wasko · 2024 [cited by examiner]
Extended European Search Report for European Patent Application No. 23173365.0 dated Mar. 6, 2024. 6 pages. [cited by applicant]
Kyriakakis, E., et al., “Hardware Assisted Clock Synchronization with the IEEE 1588-2008 Precision Time Protocol”, Real-Time Networks and Systems, ACM, Oct. 2018, pp. 51-60. [cited by applicant]
Shrivastav, V., et al., “Globally Synchronized Time via Datacenter Networks”, IEEE/ACM Transactions on Networking, vol. 27, No. 4, Aug. 2019. 16 pages. [cited by applicant]
IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. Jul. 24, 2008. IEEE Std 1588-2008 (Revision of IEEE Std 1588-2002). Technical Committee on Sensor Technology (T… [cited by applicant]