IP Library Granted Patent US 11,240,308
Granted Patent B2
US 11,240,308 · App. 16/898,216 · Granted Feb 1, 2022

Implicit discovery controller registration of non-volatile memory express (NVMe) elements in an NVME-over-fabrics (NVMe-oF) system

Inventors: Erik Smith (Douglas, MA); Joseph LaSalle White (San Jose, CA); David Black (Acton, MA); Raja Subbiah (Chennai, IN)
Assignee: DELL PRODUCTS L.P.
H04L67/1097G06F3/067G06F3/0607G06F3/0658G06F3/0659G06F3/0679H04L47/724H04L47/781H04L47/783H04L47/827
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,240,308
App. No.
16/898,216
Granted
Feb 1, 2022
Kind
B2
Abstract

Presented herein are embodiments for implicitly or indirectly registering elements of a non-volatile memory express (NVMe™) entity in an NVMe-over-Fabric (NVMe-oF) environment. In one or more embodiments, one or more interactions between an NVMe™ entity and a centralized storage fabric service component, such as part of the Link Layer Discovery Protocol (LLDP) process or the Multicast Domain Name System (mDNS) process, may be used by the centralized storage fabric service to extract information about the NVMe™ entity and automatically register it with a centralized registration datastore. In one or more embodiments, the centralized registration datastore may be used to facilitate services in the NVMe-oF system, such as discovery of NVMe™ entities, provisioning, and access control. In one or more embodiments, an implicitly registered NVMe™ entity may also subsequently explicitly register, which may include supplying additional information about the NVMe™ entity.

Claims (105)

1. A computer-implemented method for registering one or more elements of a non-volatile memory express (NVMe) entity in an NVMe-over-Fabric (NVMe-oF) system, the method comprising:

receiving, at an information handling system of the NVMe-oF system comprising a centralized storage fabric service for the NVMe-oF system, communication comprising one or more messages from an NVMe entity;

establishing a connection between the information handling system and the NVMe entity;

extracting information about the NVMe entity from communication with the NVMe entity, in which the communication from the NVMe entity is not principally intended for registration, the information comprising:

an indication of a type of NVMe entity;

transport address information and transport service information that specify a communication endpoint for an element of the NVMe entity; and

a NVMe Qualified Name (NQN); and

using at least some of the extracted information to register the NVMe entity in a centralized registration datastore by creating a registration entry associated with the NVMe entity, which registration entry is discoverable by one or more other NVMe entities in the NVMe-oF system.

2. The computer-implemented method of claim 1 , wherein the one or more messages from the NVMe entity comprise at least one of:

one or more messages communicated as part of a Link Layer Discovery Protocol (LLDP) process; and

one or more messages communicated as part of a Multicast Domain Name Service (mDNS) process.

3. The computer-implemented method of claim 1 , further comprising

receiving a discovery controller registration command (DCRC) from the NVMe entity, the DCRC comprising an identifier indicating a number of registrations the NVMe entity seeks to register with the centralized storage fabric service; and

receiving from the NVMe entity an NVMe registration for each element corresponding to the number of registrations indicated in the discovery controller registration command (DCRC); and

responsive to one of the NVMe registrations corresponding to an implicit registration entry in the centralized registration datastore, updating the implicit registration entry in the centralized registration datastore to include one or more pieces of information received as part of the NVMe registration.

4. The computer-implemented method of claim 1 wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems and the method further comprises:

receiving, at the centralized storage fabric service, a query from an NVMe entity from the plurality of NVMe entities to identify a listing of registered elements that are relevant to the NVMe entity that sent the query, in which the listing of registered elements is obtained from the centralized registration datastore of NVMe registrations; and

transmitting to the NVMe entity that sent the query, a listing of registered elements from the centralized registration datastore of registered elements that are relevant to the NVMe entity that sent the query.

5. The computer-implemented method of claim 1 :

wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems; and

wherein each NVMe registration was created for its corresponding NVMe entity:

(a) by the corresponding NVMe entity expressly requesting the centralized storage fabric service to register it in the centralized registration datastore; or

(b) by the centralized storage fabric service implicitly registering the corresponding NVMe entity; and

the method further comprises:

receiving, at the centralized storage fabric service, a request from a requesting NVMe entity for a list of one or more elements of one or more NVMe entities;

using the centralized registration datastore to generate the list of one or more elements of one or more NVMe entities for the request;

sending the generated list to the requesting NVMe entity;

receiving, from the requesting NVMe entity, a set of one or more elements from the generated list that are to be allowed access to at least one common storage element; and

responsive to receiving the set of one or more elements, forming an access control zone that grants access among the set of one or more elements.

6. The computer-implemented method of claim 1 further comprises:

responsive to a change in the centralized registration datastore, transmitting a notification of the change to one or more NVMe entities of the NVMe-oF system.

7. The computer-implemented method of claim 6 further comprises:

receiving, at the centralized storage fabric service, a request from a requesting NVMe entity for an updated list of one or more elements of one or more NVMe entities;

using information in the centralized registration datastore to generate the updated list of one or more elements of one or more NVMe entities for the request; and

sending the generated updated list to the requesting NVMe entity.

8. A system for registering one or more elements of a non-volatile memory express (NVMe) entity in an NVMe-over-Fabric (NVMe-oF) system, the system comprising:

one or more processors; and

a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

receiving, at an information handling system of the NVMe-oF system comprising a centralized storage fabric service for the NVMe-oF system, communication comprising one or more messages from an NVMe entity;

establishing a connection between the information handling system and the NVMe entity;

extracting information about the NVMe entity from communication with the NVMe entity, in which the communication from the NVMe entity is not principally intended for registration, the information comprising:

an indication of a type of NVMe entity;

transport address information and transport service information that specify a communication endpoint for an element of the NVMe entity; and

a NVMe Qualified Name (NQN); and

using at least some of the extracted information to register the NVMe entity in a centralized registration datastore by creating a registration entry associated with the NVMe entity, which registration entry is discoverable by one or more other NVMe entities in the NVMe-oF system.

9. The system of claim 8 , wherein the one or more messages from the NVMe entity comprise at least one of:

one or more messages communicated as part of a Link Layer Discovery Protocol (LLDP) process; and

one or more messages communicated as part of a Multicast Domain Name Service (mDNS) process.

10. The system of claim 8 , wherein the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

receiving a discovery controller registration command (DCRC) from the NVMe entity, the DCRC comprising an identifier indicating a number of registrations the NVMe entity seeks to register with the centralized storage fabric service; and

receiving from the NVMe entity an NVMe registration for each element corresponding to the number of registrations indicated in the discovery controller registration command (DCRC); and

responsive to one of the NVMe registrations corresponding to an implicit registration entry in the centralized registration datastore, updating the implicit registration entry in the centralized registration datastore to include one or more pieces of information received as part of the NVMe registration.

11. The system of claim 8 wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems and the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

receiving, at the centralized storage fabric service, a query from an NVMe entity from the plurality of NVMe entities to identify a listing of registered elements that are relevant to the NVMe entity that sent the query, in which the listing of registered elements is obtained from the centralized registration datastore of NVMe registrations; and

transmitting to the NVMe entity that sent the query, a listing of registered elements from the centralized registration datastore of registered elements that are relevant to the NVMe entity that sent the query.

12. The system of claim 8 :

wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems; and

wherein each NVMe registration was created for its corresponding NVMe entity:

(a) by the corresponding NVMe entity expressly requesting the centralized storage fabric service to register it in the centralized registration datastore; or

(b) by the centralized storage fabric service implicitly registering the corresponding NVMe entity; and

the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

receiving, at the centralized storage fabric service, a request from a requesting NVMe entity for a list of one or more elements of one or more NVMe entities;

using the centralized registration datastore to generate the list of one or more elements of one or more NVMe entities for the request;

sending the generated list to the requesting NVMe entity;

receiving, from the requesting NVMe entity, a set of one or more elements from the generated list that are to be allowed access to at least one common storage element; and

responsive to receiving the set of one or more elements, forming an access control zone that grants access among the set of one or more elements.

13. The system of claim 8 , wherein the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

responsive to a change in the centralized registration datastore, transmitting a notification of the change to one or more NVMe entities of the NVMe-oF system.

14. The system of claim 13 , wherein the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:

receiving, at the centralized storage fabric service, a request from a requesting NVMe entity for an updated list of one or more elements of one or more NVMe entities;

using information in the centralized registration datastore to generate the updated list of one or more elements of one or more NVMe entities for the request; and

sending the generated updated list to the requesting NVMe entity.

15. A system for registering one or more elements of a non-volatile memory express (NVMe) entity in an NVMe-over-Fabric (NVMe-oF) system, the system comprising:

a centralized storage fabric service that:

receives communication comprising one or more messages from an NVMe entity;

facilitates establishing a connection between the centralized storage fabric service and the NVMe entity;

extracts information about the NVMe entity from communication with the NVMe entity, the information comprising:

an indication of a type of NVMe entity;

transport address information and transport service information that specify a communication endpoint for an element of the NVMe entity; and

a NVMe Qualified Name (NQN); and

uses at least some of the extracted information to implicitly register the NVMe entity in a centralized registration datastore by creating a registration entry associated with the NVMe entity, which registration entry is discoverable by one or more other NVMe entities in the NVMe-oF system; and

the centralized registration datastore for storing one or more registration entries, which are discoverable by one or more other NVMe entities in the NVMe-oF system.

16. The system of claim 15 , wherein the one or more messages from the NVMe entity comprise at least one of:

one or more messages communicated as part of a Link Layer Discovery Protocol (LLDP) process; and

one or more messages communicated as part of a Multicast Domain Name Service (mDNS) process.

17. The system of claim 15 , wherein the centralized storage fabric service is further configured to perform steps comprising:

receiving a discovery controller registration command (DCRC) from the NVMe entity, the DCRC comprising an identifier indicating a number of registrations the NVMe entity seeks to register with the centralized storage fabric service; and

receiving from the NVMe entity an NVMe registration for each element corresponding to the number of registrations indicated in the discovery controller registration command (DCRC); and

responsive to one of the NVMe registrations corresponding to an implicit registration entry in the centralized registration datastore, updating the implicit registration entry in the centralized registration datastore to include one or more pieces of information received as part of the NVMe registration.

18. The system of claim 15 wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems and the centralized storage fabric service is further configured to perform steps:

receiving, at the centralized storage fabric service, a query from an NVMe entity from the plurality of NVMe entities to identify a listing of registered elements that are relevant to the NVMe entity that sent the query, in which the listing of registered elements is obtained from the centralized registration datastore of NVMe registrations; and

transmitting to the NVMe entity that sent the query, a listing of registered elements from the centralized registration datastore of registered elements that are relevant to the NVMe entity that sent the query.

19. The system of claim 15 :

wherein the centralized registration datastore comprises a plurality of NVMe registrations corresponding to a plurality of NVMe entities comprising one or more NVMe hosts and one or more NVM subsystems; and

wherein each NVMe registration was created for its corresponding NVMe entity:

(a) by the corresponding NVMe entity expressly requesting the centralized storage fabric service to register it in the centralized registration datastore; or

(b) by the centralized storage fabric service implicitly registering the corresponding NVMe entity; and

the centralized storage fabric service is further configured to perform steps:

receiving, at the centralized storage fabric service, a request from a requesting NVMe entity for a list of one or more elements of one or more NVMe entities;

using the centralized registration datastore to generate the list of one or more elements of one or more NVMe entities for the request;

sending the generated list to the requesting NVMe entity;

receiving, from the requesting NVMe entity, a set of one or more elements from the generated list that are to be allowed access to at least one common storage element; and

responsive to receiving the set of one or more elements, forming an access control zone that grants access among the set of one or more elements.

20. The system of claim 15 , wherein the centralized storage fabric service is further configured to perform steps comprising:

responsive to a change in the centralized registration datastore, transmitting a notification of the change to one or more NVMe entities of the NVMe-oF system.

Assignments (9)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053573/0535) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060333/0106 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053574/0221) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060333/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053578/0183) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060332/0864 →
RELEASE OF SECURITY INTEREST AT REEL 053531 FRAME 0108 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0371 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2020
From: SMITH, ERIK; WHITE, JOSEPH LASALLE; BLACK, DAVID; SUBBIAH, RAJA
To: DELL PRODUCTS L.P.
Reel/Frame 053701/0033 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053578/0183 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053574/0221 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053573/0535 →
SECURITY AGREEMENT Recorded Aug 18, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 053531/0108 →
Continuity (3)
Continuation In Part 16898191 · Jun 10, 2020
Continuation In Part 16820547 · Mar 16, 2020
Related Publication 20210289027A1 · Sep 16, 2021
Cited By (1)
US 12,615,298