IP Library Granted Patent US 10,936,516
Granted Patent B2
US 10,936,516 · App. 16/780,009 · Granted Mar 2, 2021

Accelerated data handling in cloud data storage system

Inventors: Mikhail Danilov (Saint Petersburg, RU); Aleksandr Rakulenko (Seattle, WA)
Assignee: EMC IP HOLDING COMPANY LLC
G06F13/20G06F12/0646G06F2212/1041
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,936,516
App. No.
16/780,009
Granted
Mar 2, 2021
Kind
B2
Abstract

The described technology is generally directed towards accelerating data handling in a cloud data storage system by using smart network interface cards (SmartNICs) at the nodes. Instead of copying data to kernel space, many input/output (I/O) operations can be handled primarily by the SmartNIC, using the SmartNIC's memory. For example, mirrored data writes can be sent directly from the SmartNIC's memory associated with the node handling the write to other nodes, without first copying the data to kernel space. Object reads can be handled at a node by having segments of the object queued, in order, in the node's associated SmartNIC's memory, and sent to a requesting client without having to be copied to the handling node's kernel space, unless low memory conditions exist in the SmartNIC's memory.

Claims (70)

1. A method, comprising,

receiving, by a smart network interface card (NIC) coupled to a node of a node cluster, a write request that identifies a first data segment and a second data segment;

determining, by the smart NIC, a handling operation for the write request;

queuing, by the smart NIC, the first data segment in a first data segment memory space of the smart NIC, and queuing the second data segment in a second data segment memory space of the smart NIC; and

outputting, la the smart NIC, the first data segment from the first data segment memory space to a receiving device.

2. The method of claim 1 , further comprising:

allocating a portion of available memory of the smart NIC in response to receiving the write request.

3. The method of claim 1 , wherein the write request comprises a descriptor, and further comprising:

providing the descriptor via the smart NIC to a storage service in the node to which the smart NIC is coupled; and

obtaining, via the smart NIC, information from the storage service that identifies the receiving device.

4. The method of claim 1 , wherein the node comprises a first node, and wherein the outputting the data segment comprises:

determining a second node of the node cluster to which the first data segment is to be copied; and

sending the first data segment to the second node.

5. The method of claim 1 , wherein the outputting the first data segment comprises:

determining that the node is to maintain a copy of the first data segment: and

sending the first data segment to a kernel space of the node.

6. The method of claim 1 , wherein the node comprises a first node, and wherein the outputting the first data segment comprises:

determining a mirroring node of the node cluster to which the first data segment is to be copied: and

sending the first data segment to the mirroring node.

7. A system, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:

receiving, at a smart network interface card (NIC), a first data segment and a second data segment identified by a write request, wherein the smart NIC is coupled to a node of a node cluster;

determining a handling operation for the write request;

queuing the first data segment in a first data segment memory space and queuing the second data segment in a second data segment memory space of the smart NIC; and

outputting the first data segment from the first data segment memory space of the smart NIC to a receiving device.

8. The system of claim 7 , wherein the operations further comprise:

receiving a read request from a client device to read the first data segment from the node cluster as part of an object read request corresponding to the first data segment and the second data segment;

queuing the first data segment and the second data segment in an ordered queue, in which the queuing is logically ordered based on sequence numbers associated with the first data segment and the second data segment; and

sending data from the ordered queue to the client device.

9. The system of claim 8 , wherein the operations further comprise:

moving a third data segment to a memory location external to the smart NIC to free up memory space in the smart NIC for the ordered queue.

10. The system of claim 7 , wherein the node comprises a first node, and wherein the outputting the first data segment comprises:

determining that the first node is to maintain a copy of the first data segment:

determining a second node and a third node of the node cluster to which the first data segment is to be mirrored: and

sending the first data segment to the second node, the third node, and a kernel space of the first node.

11. The system of claim 7 , wherein the node comprises a first node, and wherein the operations further comprise:

receiving a read request from a client device associated with a descriptor that identifies a third data segment;

providing the descriptor from the smart NIC to a storage service in the node;

obtaining information at the smart NIC from the storage service that identifies a second node of the node cluster that maintains the third data segment; and

obtaining the third data segment from the second node.

12. The system of claim 7 , wherein the write request comprises a write request to write the first data segment to the node cluster, and wherein the operations further comprise:

determining mirroring nodes of the node cluster; and

copying the first data segment to the mirroring nodes.

13. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:

receiving, at a smart network interface card (NIC), which is coupled to a node of a node cluster, a write request that identifies a first data segment and a second data segment;

determining a handling operation for the write request;

queuing the first data segment in a first data segment memory space of the smart NIC, and queuing the second data segment in a second data segment memory space of the smart NIC; and

outputting the first data segment from the first data segment memory space of the smart NIC to a receiving device.

14. The non-transitory machine-readable storage medium of claim 13 , wherein the operations further comprise:

in response to determining that there is an insufficient number of blocks for a first ordered queue of the smart NIC into which the first data segment is to be stored relative to a threshold value, moving the second data segment or a third data segment from the smart NIC to other storage of the node to which the smart NIC is coupled.

15. The non-transitory machine-readable storage medium of claim 14 , wherein the second data segment or the third data segment is moved from a selected block, and wherein the operations further comprise:

selecting the selected block based on a largest gap in sequence numbers corresponding to a first gap and a second gap, wherein the first gap is between a first current starting sequence number of first sequence numbers of the first ordered queue to a first end sequence number of the first sequence numbers of a first last available data segment in the first ordered queue, and the second gap is between a second current starting sequence number of a second sequence numbers of a second ordered queue of the smart NIC to a second end sequence number of the second sequence numbers of a second last available data segment in the second ordered queue.

16. The non-transitory machine-readable storage medium of claim 13 , wherein the node comprises a first node, and wherein the operations further comprise:

receiving a read request from a client device associated with a first descriptor that identifies the first data segment and a second descriptor that identifies the second data segment;

providing the first descriptor and the second descriptor from the smart NIC to a storage service in the node;

obtaining information at the smart NIC from the storage service that identifies a second node of the node cluster that maintains the first data segment, identifies a first sequence number of the first data segment, identifies a third node of the node cluster that maintains the second data segment, and identifies a second sequence number of the second data segment;

obtaining the first data segment from the first node and storing the first data segment in the first data segment memory space;

obtaining the second data segment from the second node and storing the second data segment in a second data segment memory space; and

sorting the first data segment and the second data segment based on the first sequence number relative to the second sequence number into ordered object parts in a queue,

wherein the outputting the data segment comprises sending the ordered object parts from the queue to the client device.

17. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise:

moving a third data segment from a third data segment memory space to a kernel space to free memory in the smart NIC for the second data segment memory space.

18. The non-transitory machine-readable storage medium of claim 17 , wherein the operations further comprise:

selecting the third data segment memory space based on a distance from the third data segment memory space within the queue to a start of the queue.

19. The non-transitory machine-readable storage medium of claim 17 , wherein the queue comprises a first queue, and wherein the operations further comprise:

selecting the third data segment memory space based on a distance from the third data segment memory space within a second queue to a start of the second queue.

20. The non-transitory machine-readable storage medium of claim 13 , wherein the write request comprises a descriptor, and wherein the operations further comprise:

providing the descriptor via the smart NIC to a storage service in the node to which the smart NIC is coupled; and

obtaining, via the smart NIC, information from the storage service that identifies the receiving device.

Assignments (13)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0081) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0441 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052851/0917) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0509 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052852/0022) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060436/0582 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053311/0169) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 060438/0742 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST AT REEL 052771 FRAME 0906 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0298 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0081 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052851/0917 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052852/0022 →
SECURITY INTEREST Recorded Jun 5, 2020
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 053311/0169 →
SECURITY AGREEMENT Recorded May 28, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052771/0906 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2020
From: DANILOV, MIKHAIL; RAKULENKO, ALEKSANDR
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 051700/0982 →