IP Library › Granted Patent US 12,222,750
Granted Patent B2
US 12,222,750 · App. 18/198,150 · Granted Feb 11, 2025

Timestamp alignment across multiple computing nodes

Inventors: Mark Bordogna (Andover, MA); Jonathan A. Robinson (Portland, OR)
Assignee: Intel Corporation
G06F1/14G06F1/12G06F9/542H04L43/106
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,222,750
App. No.
18/198,150
Granted
Feb 11, 2025
Kind
B2
Abstract

Examples described herein relate to multiple processor nodes which are physically separate with interfaces to a common network interface. A local processor can run a timing recovery algorithm, and tune the master timer to align with the network domain to cause the master timer and network timing domains to be in the same domain. A common master timer can be used to align time stamps of independent processor nodes. A processor node can use the common master timer as a reference and the processor does not need to communicate with another processor to synchronize its timer.

Claims (43)

1. An apparatus comprising:

a network interface device comprising:

a first interface to a first processor and

circuitry to receive a time stamp from the first processor by the first interface, wherein:

receipt of the time stamp from the first processor by the first interface is to cause the circuitry to provide, to the first processor, a time stamp value based on a clock signal from a clock source corresponding to the received time stamp from the first processor and

the first processor is to adjust a time stamp counter value based, at least in part, on the time stamp value based on the clock signal corresponding to the received time stamp from the first processor.

2. The apparatus of claim 1 , wherein the first processor is to adjust a time stamp counter value based, at least in part, on the time stamp value based on the clock signal corresponding to the received time stamp from the first processor based on a linear relationship.

3. The apparatus of claim 1 , wherein the network interface device comprises

a second interface to a second processor and

circuitry is to receive a time stamp from the second processor by the second interface.

4. The apparatus of claim 3 , wherein

the second processor is to adjust a time stamp counter value based, at least in part, on the time stamp value based on the clock signal corresponding to a received time stamp from the second processor.

5. The apparatus of claim 1 , wherein

the first interface is to operate consistent with Peripheral Component Interconnect express (PCIe).

6. A method comprising:

a first processor node providing first and second time stamp counter (TSC) values to a network interface device to cause the network interface device to provide a first TSC value-primary timer value pair to the first processor node and provide a second TSC value-primary timer value pair to the first processor node;

the first processor node determining a first offset of the first TSC for the first processor node relative to the primary timer based on the first and second TSC value-primary timer value pairs received from the network interface device; and

the first processor node adjusting values of the TSC for the first processor node based on the first offset.

7. The method of claim 6 , comprising:

a second processor node providing first and second TSC values to the network interface device to cause the network interface device to provide a first TSC value-primary timer value pair to the second processor node and provide a second TSC value-primary timer value pair to the second processor node;

the second processor node determining a second offset of the TSC values for the second processor node relative to the primary timer; and

the second processor node adjusting the TSC values for the second processor node based on the second offset.

8. The method of claim 6 , wherein the primary timer is based on a network source based on IEEE 1588 or a global positioning system (GPS) signal.

9. The method of claim 6 , wherein the first offset comprises a linear offset.

10. The method of claim 7 , wherein the first offset comprises a linear offset and the second offset comprises a linear offset.

11. The method of claim 6 , wherein the first processor node utilizes a first crystal oscillator to generate the first and second TSC values for the first processor node.

12. The method of claim 7 , wherein the second processor node utilizes a second crystal oscillator to generate the first and second TSC values for the second processor node.

13. An apparatus comprising:

a first processor node comprising a first processor and a first time stamp counter (TSC) source, wherein:

by a copy of a first value of the first TSC source to a network interface device, the first processor node is to initiate and cause the network interface device to transfer a primary time stamp value recorded for the first value of the first TSC source from the network interface device to the first processor node.

14. The apparatus of claim 13 , wherein:

the first processor node is to determine a first offset for the first TSC source based, at least, in part on the primary time stamp values recorded for the first and second values of the first TSC source and

the first processor comprises one or more of: a central processing unit (CPU), core, graphics processing unit (GPU), field programmable gate array (FPGA), or application specific integrated circuit (ASIC).

15. The apparatus of claim 14 , wherein the first processor node is to execute an virtualized execution environment with time stamps that are synchronized based on the first offset.

16. The apparatus of claim 13 , comprising:

a second processor node comprising a second processor and a second TSC source, wherein the second processor comprises one or more of: a central processing unit (CPU), core, graphics processing unit (GPU), field programmable gate array (FPGA), or application specific integrated circuit (ASIC) and wherein the second processor node is to determine a second offset for a second TSC source based, at least, in part on the primary time stamp values recorded for the first and second values of the second TSC source.

17. The apparatus of claim 13 , comprising a computer, a rack, or a data center and wherein the computer, the rack, or the data center comprises the first processor node.

18. The apparatus of claim 13 , wherein at least one primary time stamp value is based on a network source based on IEEE 1588 or a global positioning system (GPS) signal.

19. The apparatus of claim 13 , wherein the first processor node is to use a first crystal oscillator to generate the first value of the first TSC source.

20. The apparatus of claim 1 , wherein:

the first processor comprises one or more of: a central processing unit (CPU), core, graphics processing unit (GPU), field programmable gate array (FPGA), or application specific integrated circuit (ASIC).

21. The apparatus of claim 3 , wherein:

the second processor comprises one or more of: a central processing unit (CPU), core, graphics processing unit (GPU), field programmable gate array (FPGA), or application specific integrated circuit (ASIC).

Continuity (3)
Continuation 16726684 · Dec 24, 2019
Provisional Application 62814203 · Mar 5, 2019
Related Publication 20230367362A1 · Nov 16, 2023
References Cited (61)
US 5535193A · Zhang et al. · 1996 [cited by applicant]
US 5579513A · Strohmer · 1996 [cited by applicant]
US 6556832B1 · Soliman · 2003 [cited by applicant]
US 7103072B1 · Sloan et al. · 2006 [cited by applicant]
US 7590151B2 · Middleton et al. · 2009 [cited by applicant]
US 7903694B2 · Defrance et al. · 2011 [cited by applicant]
US 9003220B2 · Chandhoke et al. · 2015 [cited by applicant]
US 9058135B1 · Schumacher et al. · 2015 [cited by applicant]
US 11693448B2 · Bordogna · 2023 [cited by examiner]
US 20010055311A1 · Trachewsky et al. · 2001 [cited by applicant]
US 20020039370A1 · Elliot · 2002 [cited by applicant]
US 20030117899A1 · Eidson · 2003 [cited by applicant]
US 20030177154A1 · Vrancic · 2003 [cited by applicant]
US 20040049706A1 · Strong · 2004 [cited by applicant]
US 20040062278A1 · Hadzic et al. · 2004 [cited by applicant]
US 20050228902A1 · Lienhart et al. · 2005 [cited by applicant]
US 20070038999A1 · Millington · 2007 [cited by applicant]
US 20070266119A1 · Ohly · 2007 [cited by applicant]
US 20080270800A1 · Krober et al. · 2008 [cited by applicant]
US 20100103781A1 · Rai et al. · 2010 [cited by applicant]
US 20100118895A1 · Radulescu · 2010 [cited by applicant]
US 20100158183A1 · Hadzic et al. · 2010 [cited by applicant]
US 20100177763A1 · Stok et al. · 2010 [cited by applicant]
US 20110276994A1 · Higgins et al. · 2011 [cited by applicant]
US 20120030495A1 · Chandhoke et al. · 2012 [cited by applicant]
US 20130125118A1 · Niesser et al. · 2013 [cited by applicant]
US 20130301635A1 · Hollabaugh et al. · 2013 [cited by applicant]
US 20130318390A1 · Takehara · 2013 [cited by applicant]
US 20140098684A1 · Diab et al. · 2014 [cited by applicant]
US 20140143582A1 · Kindred et al. · 2014 [cited by applicant]
US 20140313289A1 · Kim et al. · 2014 [cited by applicant]
US 20140355628A1 · Akhlaq et al. · 2014 [cited by applicant]
US 20150163049A1 · Nors · 2015 [cited by applicant]
US 20150277989A1 · Laborde et al. · 2015 [cited by applicant]
US 20160062936A1 · Brassac · 2016 [cited by examiner]
US 20160072883A1 · Long · 2016 [cited by examiner]
US 20160112182A1 · Karnes · 2016 [cited by applicant]
US 20160170439A1 · Aweya · 2016 [cited by applicant]
US 20160170440A1 · Aweya · 2016 [cited by applicant]
US 20160231770A1 · Mallada · 2016 [cited by examiner]
US 20170237512A1 · Galea et al. · 2017 [cited by applicant]
US 20170315582A1 · Zhang et al. · 2017 [cited by applicant]
US 20170330278A1 · Radulescu · 2017 [cited by applicant]
US 20180227080A1 · Stein et al. · 2018 [cited by applicant]
US 20190045475A1 · Kasichainula · 2019 [cited by applicant]
US 20190140815A1 · Faist et al. · 2019 [cited by applicant]
US 20190208381A1 · Booij et al. · 2019 [cited by applicant]
US 20190332137A1 · Benjamini et al. · 2019 [cited by applicant]
US 20200033906A1 · Maddukuri et al. · 2020 [cited by applicant]
US 20200133330A1 · Bordogna et al. · 2020 [cited by applicant]
US 20220337683A1 · Biederman et al. · 2022 [cited by applicant]
KR 1020200073983A · 2020 [cited by applicant]
Final Office Action for U.S. Appl. No. 16/726,684, Mailed Sep. 9, 2022, 13 pages. [cited by applicant]
First Office Action for U.S. Appl. No. 16/726,684, Mailed Mar. 18, 2022, 11 pages. [cited by applicant]
Garner, Geoffrey M., “IEEE 1588 Version 2”, ISPCS Ann Arbor '08, Sep. 24, 2008, 89 pages. [cited by applicant]
Intel, “Intel 64 and IA-32 Architectures Software Developer's Manual, Combined vols. 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D and 4, Section 17.17 Time-Stamp Counter”, vol. 3B-17-41, May 2020, pp. 3431-3433, 3 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Patent Application No. PCT/US22/20813, Mailed Jun. 24, 2022, 14 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/726,684, Mailed Feb. 16, 2023, 5 pages. [cited by applicant]
Sebastian Stieber et al., ‘Accurate Sample Time Reconstruction of Inertial FIFO Data’, MDPI, sensor, Dec. 13, 2017, doi: 10.3390/s17122894. [cited by applicant]
Solomon, Richard, “Express Yourself—Spring Ahead”, SYNOPSYS, https://blogs.synopsys.com/expressyourself/2014/03/09/spring-ahead/, posted Mar. 9, 2014, 4 pages. [cited by applicant]
First Office Action for U.S. Appl. No. 17/500,576, Mailed Jun. 5, 2024, 16 pages. [cited by applicant]
Cited By (1)
US 12,743,122