IP Library Granted Patent US 11,314,666
Granted Patent B2
US 11,314,666 · App. 17/025,691 · Granted Apr 26, 2022

Systems and methods for optimizing clock distribution in NVMe storage enclosures

Inventors: Dor Aharony (Newton, MA); Stephen Strickland (Foxborough, MA); Gary W. Smith (Cumberland, RI)
Assignee: Dell Products, LP
G06F13/1668G06F1/04G06F1/10G06F1/1658G06F1/187G06F1/206G06F13/4291G06F2213/0026
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,314,666
App. No.
17/025,691
Granted
Apr 26, 2022
Kind
B2
Abstract

A non-volatile memory express (NVMe) based storage system may include a processor; an input/output (I/O) port operatively coupled to a data network; a drive interface board (DIB), including: a PCIe switch communicatively coupled to the processor to receive signals from the processor to a plurality of NVMe storage devices; and a source clock communicatively coupled to the PCIe switch; a connector communicatively coupling the DIB to relay clock signals to a plurality of NVMe drives at a drive mid-plane baseboard; the drive mid-plane baseboard communicatively coupled to the source clock, the drive mid-plane baseboard including: a clock distribution module to receive a source clock signal from the source clock, the clock distribution modules placed at the drive baseboard to distribute the clock signals to the plurality of NVMe storage devices.

Claims (54)

1. A non-volatile memory express (NVMe) based storage system, comprising:

a processor;

an input/output (I/O) port operable to couple to a data network;

a drive interface board (DIB), including:

a PCIe switch communicatively coupled to the processor to receive signals from the processor to a plurality of NVMe storage devices; and

a source clock communicatively coupled to the PCIe switch;

a connector communicatively coupling the DIB and configured to relay clock signals to a plurality of NVMe drives that are operatively coupled to a drive mid-plane baseboard; and

the drive mid-plane baseboard communicatively coupled to the source clock and the PCIe switch, the drive mid-plane baseboard including:

a clock distribution module to receive a source clock signal from the source clock, the clock distribution module placed at the drive baseboard to distribute the clock signals to the plurality of NVMe storage devices.

2. The NVMe-based storage system of claim 1 , further comprising:

the drive mid-plane baseboard including a plurality of vents;

a cooling system including a plurality of fans placed between the drive interface board and the drive mid-plane baseboard; and

the connector communicatively coupling the source clock to the distribution module on the drive mid-plane baseboard comprising clock signal pins between the source clock and the clock distribution module.

3. The NVMe-based storage system of claim 1 , wherein the clock distribution modules operate as dependent clocks to the source clock.

4. The NVMe-based storage system of claim 1 , wherein the PCIe switch receives source clock signals from the source clock to time data signal distribution at the drive interface board to the NVMe storage devices.

5. The NVMe-based storage system of claim 1 , wherein the drive mid-plane baseboard includes a plurality of clock distribution modules coupled to the drive mid-plane baseboard.

6. The NVMe-based storage system of claim 1 , further comprising:

additional drive interface boards, PCIe switches, and source clocks to interface with the plurality of NVMe storage devices and a plurality of clock distribution modules.

7. The NVMe-based storage system of claim 1 , wherein the clock distribution module relays clock signals from the source clock to a group of NVMe memory devices.

8. A method of assembling an NVMe-based information handling system, comprising:

operatively coupling a processor on a link controller card (LCC) with a data port having network connectivity, the LCC including:

a PCIe switch communicatively coupled to the processor to receive signals from the processor and to transmit those signals to a plurality of NVMe storage devices operatively coupled to a drive mid-plane baseboard;

a source clock communicatively coupled to the PCIe switch; and

a connector communicatively coupled to the PCIe switch to relay signals to the plurality of NVMe storage devices at the drive mid-plane baseboard;

coupling the plurality of NVMe storage devices to the drive mid-plane baseboard, the drive mid-plane baseboard including:

a clock distribution board communicatively coupled to the source clock via the connector; and

a clock distribution module coupled to the clock distribution board to receive a source clock signal from the source clock, the clock distribution module to distribute clocking signals to the plurality of NVMe storage devices.

9. The method of claim 8 , further comprising:

forming a cooling system including a plurality of fans placed between the LCC and the drive mid-plane baseboard;

the connector communicatively coupling the source clock to the clock distribution module via a PCB connection board passing the plurality of fans to the connector to the mid-plane baseboard.

10. The method of claim 8 , further comprising:

the clock distribution module is operatively coupled to a second clock distribution module on the drive mid-plane baseboard and operates as a dependent clock to the source clock.

11. The method of claim 8 , wherein the connector has a reduced number of pins for clocking signals from the source clock to the clock distribution module as compared to the clocking signals provided by the clock distribution module to the plurality of NVMe storage devices.

12. The method of claim 8 , wherein the source clock in interfaces with the connector, the connector further comprising:

two dedicated ports for each clock distribution module.

13. The method of claim 8 , further comprising:

the LCC with an additional processor, an additional PCIe switch, and additional source clock to interface with a plurality of clock distribution modules.

14. The method of claim 8 , further comprising a plurality of clock distribution modules where each clock distribution module relays signals from the source clock to a group of NVMe memory devices.

15. A non-volatile memory express (NVMe) based storage system, comprising:

a storage access module including a processor and an input/output (I/O) port;

a drive interface board including:

a first electronic oscillator communicatively coupled to the processor;

a PCIe switch communicatively coupled to the first electronic oscillator to receive a clock signal from the first electronic oscillator;

a connector communicatively coupled to the electronic oscillator to relay clock signals to a plurality of clock distribution modules at a drive mid-plane baseboard and the connector communicatively coupled to the PCIe switch to relay data signals between the processor and a plurality of NVMe storage devices at the drive mid-plane baseboard;

a cooling system including a plurality of fans placed between the drive interface board and the drive mid-plane baseboard;

a connection board operatively coupling the drive interface board to the connector at the drive mid-plane baseboard;

wherein the drive mid-plane baseboard is communicatively coupled to the electronic oscillator, the drive mid-plane baseboard including:

data and power connection to the plurality of NVMe storage devices; and

each of the plurality of clock distribution modules to relay clock signals to a group of the NVMe storage devices.

16. The NVMe-based storage system of claim 15 , wherein the plurality of clock distribution modules operate as dependent clocks to the first electronic oscillator.

17. The NVMe-based storage system of claim 15 , wherein one of the clock distribution modules operates as a source clock to the first electronic oscillator.

18. The NVMe-based storage system of claim 15 , wherein a connector size is reduced due to a reduced number of clock signal pins.

19. The NVMe-based storage system of claim 15 , further comprising an additional processor, an additional PCIe switch, and an additional connector to interface with a plurality of clock signal pins and the plurality of clock distribution modules.

20. The NVMe-based storage system of claim 15 , further comprising a plurality of vents in the mid-plane baseboard to provide airflow from the plurality of fans to the plurality of NVMe storage devices where the connector is streamlined to facilitate airflow through the plurality of vents.

Assignments (9)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (054475/0523) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
Reel/Frame 060332/0664 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (054475/0434) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
Reel/Frame 060332/0740 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (054475/0609) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
Reel/Frame 062021/0570 →
RELEASE OF SECURITY INTEREST AT REEL 054591 FRAME 0471 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0463 →
SECURITY INTEREST Recorded Nov 18, 2020
From: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 054475/0609 →
SECURITY INTEREST Recorded Nov 18, 2020
From: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 054475/0434 →
SECURITY INTEREST Recorded Nov 18, 2020
From: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 054475/0523 →
SECURITY AGREEMENT Recorded Nov 13, 2020
From: EMC IP HOLDING COMPANY LLC; DELL PRODUCTS L.P.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 054591/0471 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 18, 2020
From: AHARONY, DOR; STRICKLAND, STEPHEN; SMITH, GARY W.
To: DELL PRODUCTS, LP
Reel/Frame 053821/0241 →