IP Library Granted Patent US 11,106,607
Granted Patent B1
US 11,106,607 · App. 16/836,341 · Granted Aug 31, 2021

NUMA-aware storage system

Inventors: Gary Benedict Kotzur (Austin, TX); William Emmett Lynn (Round Rock, TX); Kevin Thomas Marks (Georgetown, TX); Chandrashekar Nelogal (Round Rock, TX); James Peter Giannoules (Round Rock, TX); Austin Patrick Bolen (Austin, TX)
Assignee: Dell Products L.P.
G06F13/1668G06F13/4022G06F13/4027
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,106,607
App. No.
16/836,341
Granted
Aug 31, 2021
Kind
B1
Abstract

A NUMA-aware storage system including a first processing subsystem coupled to a first memory subsystem, and a second processing subsystem coupled to a second memory subsystem. A first NTB subsystem connected to the first processing subsystem presents itself as a first storage device, identifies first data transfer operations directed to the first memory subsystem and, in response, claims those first data transfer operations and provides them directly to the first processing subsystem. A second NTB subsystem connected to the second processing subsystem presents itself as a second storage device, identifies second data transfer operations directed to the second memory subsystem and, in response, claims those second data transfer operations and provides them directly to the second processing subsystem. A storage controller system receives a command from either the first or second processing subsystem via the first or second NTB subsystem and, in response, transmits that command to a storage system.

Claims (59)

1. A Non-Uniform Memory Access (NUMA)-aware storage system, comprising:

a first processing subsystem coupled to a first memory subsystem;

a second processing subsystem coupled to a second memory subsystem;

a storage system; and

a storage controller system that is coupled to the storage system and that includes:

a first Non-Transparent Bridge (NTB) subsystem that is connected to the first processing subsystem and that is configured to:

present the first NTB subsystem to the first processing subsystem as a first storage device; and

identify first data transfer operations directed to the first memory subsystem and, in response, claim those first data transfer operations and provide them directly to the first processing subsystem;

a second NTB subsystem that is connected to the second processing subsystem and that is configured to:

present the second NTB subsystem to the second processing subsystem as a second storage device; and

identify second data transfer operations directed to the second memory subsystem and, in response, claim those second data transfer operations and provide them directly to the second processing subsystem; and

wherein the storage controller system is configured to:

receive a command from either the first processing subsystem via the first NTB subsystem or the second processing subsystem via the second NTB subsystem and, in response, transmit that command to the storage system.

2. The system of claim 1 , wherein the storage controller system includes:

a switch device that includes the first NTB subsystem and the second NTB subsystem; and

a storage controller device that is coupled to the switch device.

3. The system of claim 1 , wherein the storage controller system is configured to:

spoof a namespace that identifies a storage space provided by the storage system to the first processing subsystem and the second processing subsystem.

4. The system of claim 1 , wherein the storage system includes a plurality of Non-Volatile Memory Express (NVMe) storage devices, the first NTB subsystem is presented to the first processing subsystem as a first NVMe storage device, and the second NTB subsystem is presented to the second processing subsystem as a second NVMe storage device.

5. The system of claim 1 , wherein the first processing subsystem is coupled to the second processing subsystem by processing subsystem interconnect, and wherein the first data transfer operations are provided directly to the first processing subsystem without using the processing subsystem interconnect, and the second data transfer operations are provided directly to the second processing subsystem without using the processing subsystem interconnect.

6. The system of claim 1 , wherein the storage controller device is configured to:

program the first NTB subsystem to claim first memory addresses that are included in the first memory subsystem; and

program the second NTB subsystem to claim second memory addresses that are included in the second memory subsystem.

7. An Information Handling System (IHS), comprising:

a first Non-Transparent Bridge (NTB) subsystem that is connected to a first processing subsystem and that is configured to:

present the first NTB subsystem to the first processing subsystem as a first storage device; and

identify first data transfer operations directed to a first memory subsystem that is connected to the first processing subsystem and, in response, claim those first data transfer operations and provide them directly to the first processing subsystem;

a second NTB subsystem that is connected to a second processing subsystem and that is configured to:

present the second NTB subsystem to the second processing subsystem as a second storage device; and

identify second data transfer operations directed to a second memory subsystem that is connected to the second processing subsystem and, in response, claim those second data transfer operations and provide them directly to the second processing subsystem;

a controller processing system that is coupled to the first NTB subsystem and the second NTB subsystem; and

a controller memory system that is coupled to the controller processing system and that includes instructions that, when executed by the controller processing system, cause the controller processing system to provide a storage controller engine that is configured to:

receive a command from either the first processing subsystem via the first NTB subsystem or the second processing subsystem via the second NTB subsystem and, in response, transmit that command to a storage system.

8. The IHS of claim 7 , further comprising:

a switch device that includes the first NTB subsystem and the second NTB subsystem; and

a storage controller device that is coupled to the switch device and that includes the controller processing system and the controller memory system.

9. The IHS of claim 7 , wherein the storage controller engine is configured to:

spoof a namespace that identifies a storage space provided by the storage system to the first processing subsystem and the second processing subsystem.

10. The IHS of claim 7 , wherein the storage system includes a plurality of Non-Volatile Memory Express (NVMe) storage devices, the first NTB subsystem is presented to the first processing subsystem as a first NVMe storage device, and the second NTB subsystem is presented to the second processing subsystem as a second NVMe storage device.

11. The IHS of claim 7 , wherein the first processing subsystem is coupled to the second processing subsystem by processing subsystem interconnect, and wherein the first data transfer operations are provided directly to the first processing subsystem without using the processing subsystem interconnect, and the second data transfer operations are provided directly to the second processing subsystem without using the processing subsystem interconnect.

12. The IHS of claim 7 , wherein the storage controller engine is configured to:

program the first NTB subsystem to claim first memory addresses that are included in the first memory subsystem; and

program the second NTB subsystem to claim second memory addresses that are included in the second memory subsystem.

13. The IHS of claim 7 , wherein the storage subsystem is a Redundant Array of Independent Disks (RAID) storage subsystem including a plurality of RAID data storage devices.

14. A method for storing data in a Non-Uniform Memory Access (NUMA)-enabled computing device, comprising:

presenting, by a first Non-Transparent Bridge (NTB) subsystem, the first NTB subsystem to a first processing subsystem as a first storage device; and

identifying, by the first NTB subsystem, first data transfer operations directed to a first memory subsystem that is connected to the first processing subsystem and, in response, claiming those first data transfer operations and providing them directly to the first processing subsystem;

presenting, by a second NTB subsystem, the second NTB subsystem to a second processing subsystem as a second storage device; and

identifying, by the second NTB subsystem, second data transfer operations directed to a second memory subsystem that is connected to the second processing subsystem and, in response, claiming those second data transfer operations and providing them directly to the second processing subsystem;

receiving, by a storage controller engine, a command from either the first processing subsystem via the first NTB subsystem or the second processing subsystem via the second NTB subsystem and, in response, transmitting that command to a storage system.

15. The method of claim 14 , wherein the first NTB subsystem and the second NTB subsystem are included in a switch device, and wherein the storage controller engine is included in a storage controller device that is coupled to the switch device.

16. The method of claim 14 , further comprising:

spoofing, by the storage controller engine, a namespace that identifies a storage space provided by the storage system to the first processing subsystem and the second processing subsystem.

17. The method of claim 14 , wherein the storage system includes a plurality of Non-Volatile Memory Express (NVMe) storage devices, the first NTB subsystem is presented to the first processing subsystem as a first NVMe storage device, and the second NTB subsystem is presented to the second processing subsystem as a second NVMe storage device.

18. The method of claim 14 , wherein the first processing subsystem is coupled to the second processing subsystem by processing subsystem interconnect, and wherein the first data transfer operations are provided directly to the first processing subsystem without using the processing subsystem interconnect, and the second data transfer operations are provided directly to the second processing subsystem without using the processing subsystem interconnect.

19. The method of claim 14 , further comprising:

programming, by the storage controller engine, the first NTB subsystem to claim first memory addresses that are included in the first memory subsystem; and

programming, by the storage controller engine, the second NTB subsystem to claim second memory addresses that are included in the second memory subsystem.

20. The method of claim 14 , wherein the storage subsystem is a Redundant Array of Independent Disks (RAID) storage subsystem including a plurality of RAID data storage devices.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0081) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0441 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0917) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0509 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052852/0022) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0582 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST AT REEL 052771 FRAME 0906 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0298 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2021
From: KOTZUR, GARY BENEDICT; LYNN, WILLIAM EMMETT; MARKS, KEVIN THOMAS; NELOGAL, CHANDRASHEKAR; GIANNOULES, JAMES PETER; BOLEN, AUSTIN PATRICK
To: DELL PRODUCTS L.P.
Reel/Frame 056863/0957 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052852/0022 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0917 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0081 →
SECURITY AGREEMENT Recorded May 28, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052771/0906 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →