IP Library › Granted Patent US 11,301,163
Granted Patent B2
US 11,301,163 · App. 16/911,736 · Granted Apr 12, 2022

NOF-based read control method, apparatus, and system

Inventors: Shengguang Zhang (Hangzhou, CN); Haiyan Hu (Hangzhou, CN); Xinqian Zheng (Shenzhen, CN)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
G06F3/0655G06F3/061G06F3/0679G06F13/4221G06F2213/0026
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,301,163
App. No.
16/911,736
Granted
Apr 12, 2022
Kind
B2
Abstract

A NOF-based read control method, apparatus, and system belong to the field of networked storage. The method includes: receiving, by a NOF engine by using a communication link, a read request sent by a host; sending at least one read command to an NVMe hard disk based on the read request; and when congestion occurs on the communication link, generating a congestion flag corresponding to the communication link, and sending the congestion flag to the NVMe hard disk, where the congestion flag is used to instruct the NVMe hard disk to suspend processing of the read command corresponding to the communication link.

Claims (77)

1. A read control method comprising:

receiving, by a non-volatile memory express over fabric (NOF) engine, using a communication link, a read request sent by a host;

sending, by the NOF engine, at least one read command to a non-volatile memory express (NVMe) hard disk according to the read request;

generating, by the NOF engine in response to congestion occurring on the communication link, a congestion flag corresponding to the communication link; and

sending, by the NOF engine, the congestion flag to the NVMe hard disk, wherein the congestion flag instructs the NVMe hard disk to suspend processing of the read command corresponding to the communication link.

2. The method according to claim 1 , wherein the sending, by the NOF engine, the congestion flag to the NVMe hard disk comprises:

determining, by the NOF engine, a submission queue corresponding to the communication link, wherein the submission queue stores a plurality of Submission Queue Entry (SQE) commands in the submission queue corresponding to the communication link, and wherein a first SQE command comprises at least one of a read command or a write command; and

sending, by the NOF engine, in response to receiving a command read request that is of the NVMe hard disk and that is for the submission queue, a second SQE command carrying the congestion flag to the NVMe hard disk.

3. The method according to claim 2 , wherein the method further comprises performing, after the sending, to the NVMe hard disk, the second SQE command:

sending, by the NOF engine, an idle frame to the host at an interval using the communication link;

generating, by the NOF engine, in response to receiving an acknowledgment signal of the idle frame, a congestion release flag corresponding to the communication link; and

sending, by the NOF engine, the congestion release flag to the NVMe hard disk, wherein the congestion release flag instructs the NVMe hard disk to resume the processing of the read command corresponding to the communication link.

4. The method according to claim 3 , wherein the sending, by the NOF engine, the congestion release flag to the NVMe hard disk comprises:

determining, by the NOF engine, the submission queue corresponding to the communication link, wherein the submission queue stores the plurality of SQE commands corresponding to the communication link; and

sending, by the NOF engine, in response to receiving a command read request that is of the NVMe hard disk and that is for the submission queue, a third SQE command carrying the congestion release flag to the NVMe hard disk.

5. The method according to claim 3 , wherein the sending, by the NOF engine, the congestion release flag to the NVMe hard disk comprises:

determining, by the NOF engine, the submission queue corresponding to the communication link, wherein the submission queue stores the plurality of SQE commands corresponding to the communication link; and

sending, by the NOF engine, to the NVMe hard disk, a register write command pointing to a second user-defined address, wherein the second user-defined address is an address that stores the congestion release flag, and wherein the register write command carries an identifier of the submission queue and the congestion release flag.

6. The method according to claim 3 , wherein the sending, by the NOF engine, the congestion release flag to the NVMe hard disk comprises:

determining, by the NOF engine, the submission queue corresponding to the communication link, wherein the submission queue stores the plurality of SQE commands corresponding to the communication link; and

sending, by the NOF engine, a management command containing second user-defined content to the NVMe hard disk, wherein the second user-defined content carries an identifier of the submission queue and the congestion release flag.

7. The method according to claim 1 , wherein the sending, by the NOF engine, the congestion flag to the NVMe hard disk comprises:

determining, by the NOF engine, a submission queue corresponding to the communication link, wherein the submission queue stores a Submission Queue Entry (SQE) command corresponding to the communication link; and

sending, by the NOF engine, to the NVMe hard disk, a register write command pointing to a first user-defined address, wherein the first user-defined address is an address that stores the congestion flag, and wherein the register write command carries an identifier of the submission queue and the congestion flag.

8. The method according to claim 1 , wherein the sending, by the NOF engine, the congestion flag to the NVMe hard disk comprises:

determining, by the NOF engine, a submission queue corresponding to the communication link, wherein the submission queue stores a Submission Queue Entry (SQE) command corresponding to the communication link; and

sending, by the NOF engine, a management command containing first user-defined content to the NVMe hard disk, wherein the first user-defined content carries an identifier of the submission queue and the congestion flag.

9. A non-volatile memory express over fabric (NOF)-based read control method, comprising:

receiving, by a NOF engine having a data buffer corresponding to a communication link, using the communication link, a read request sent by a host, wherein the read request requests to read data of a first data amount, and wherein the first data amount exceeds a size of the data buffer corresponding to the communication link;

splitting, by the NOF engine, the read request into K read commands, wherein the read command requests to read data of a second data amount, and wherein the second data amount is less than the first data amount;

sending, by the NOF engine, m read commands to a non-volatile memory express (NVMe) hard disk, wherein m≤K, and a product of m and the second data amount is not greater than the size of the data buffer corresponding to the communication link; and

suspending, by the NOF engine in response to congestion occurring on the communication link, sending of a remaining read command to the NVMe hard disk.

10. The method according to claim 9 , wherein the method further comprises performing, after the suspending the sending of the remaining read command to the NVMe hard disk:

receiving, by the NOF engine, data read by the NVMe hard disk according to the m read commands;

storing, by the NOF engine, the data in the data buffer corresponding to the communication link; and

sending, by the NOF engine in response to the congestion on the communication link being released, the data in the data buffer to the host using the communication link.

11. The method according to claim 10 , wherein the method further comprises performing, after the sending the data in the data buffer to the host:

continuing, by the NOF engine, to send n read commands to the NVMe hard disk, wherein n≤K−m, and wherein a product of n and the second data amount is not greater than the size of the data buffer corresponding to the communication link.

12. A non-volatile memory express over fabric (NOF) engine, comprising:

at least one processor; and

a memory, storing a program to be executed by the at least one processor, the program including instructions for:

receiving, using a communication link, a read request sent by a host;

sending at least one read command to a non-volatile memory express (NVMe) hard disk according to the read request;

generating a congestion flag corresponding to the communication link in response to congestion occurring on the communication link; and

sending the congestion flag to the NVMe hard disk, wherein the congestion flag instructs the NVMe hard disk to suspend processing of the read command corresponding to the communication link.

13. The NOF engine according to claim 12 , wherein the instructions for sending the congestion flag to the NVMe hard disk include instructions for:

determining a submission queue corresponding to the communication link, wherein the submission queue stores a plurality of Submission Queue Entry (SQE) commands in the submission queue corresponding to the communication link, and wherein a first SQE command comprises at least one of a read command or a write command; and

sending, in response to receiving a command read request that is of the NVMe hard disk and that is for the submission queue, a second SQE command carrying the congestion flag to the NVMe hard disk.

14. The NOF engine according to claim 13 , wherein the program further includes instructions for performing, after the sending, to the NVMe hard disk, the second SQE command:

sending, by the NOF engine, an idle frame to the host at an interval using the communication link;

generating, by the NOF engine, in response to receiving an acknowledgment signal of the idle frame, a congestion release flag corresponding to the communication link; and

sending, by the NOF engine, the congestion release flag to the NVMe hard disk, wherein the congestion release flag instructs the NVMe hard disk to resume the processing of the read command corresponding to the communication link.

15. The NOF engine according to claim 14 , wherein the instructions for sending the congestion release flag to the NVMe hard disk include instructions for:

determining, by the NOF engine, the submission queue corresponding to the communication link, wherein the submission queue stores the plurality of SQE commands corresponding to the communication link; and

performing at least one of:

sending, in response to receiving a command read request that is of the NVMe hard disk and that is for the submission queue, a third SQE command carrying the congestion release flag to the NVMe hard disk;

sending, to the NVMe hard disk, a register write command pointing to a second user-defined address, wherein the second user-defined address is an address that stores the congestion release flag, and wherein the register write command carries an identifier of the submission queue and the congestion release flag; or

sending, a management command containing second user-defined content to the NVMe hard disk, wherein the second user-defined content carries the identifier of the submission queue and the congestion release flag.

16. The NOF engine according to claim 12 , wherein the instructions for sending the congestion flag to the NVMe hard disk include instructions for:

determining a submission queue corresponding to the communication link, wherein the submission queue stores a Submission Queue Entry (SQE) command corresponding to the communication link; and

sending, by the NOF engine, to the NVMe hard disk, a register write command pointing to a first user-defined address, wherein the first user-defined address is an address that stores the congestion flag, and wherein the register write command carries an identifier of the submission queue and the congestion flag.

17. The NOF engine according to claim 12 , wherein the instructions for sending, by the NOF engine, the congestion flag to the NVMe hard disk include instructions for:

determining, by the NOF engine, a submission queue corresponding to the communication link, wherein stores a Submission Queue Entry (SQE) command corresponding to the communication link; and

sending, by the NOF engine, a management command containing first user-defined content to the NVMe hard disk, wherein the first user-defined content carries an identifier of the submission queue and the congestion flag.

18. A non-volatile memory express over fabric (NOF) engine, comprising:

at least one processor; and

a memory, wherein the memory comprises a data buffer corresponding to a communication link, wherein the memory stores a program to be executed by the at least one processor, the program including instructions for:

receiving, using the communication link, a read request sent by a host, wherein the read request requests to read data of a first data amount, and wherein the first data amount exceeds a size of the data buffer of the communication link;

splitting the read request into K read commands, wherein the read command requests to read data of a second data amount, and wherein the second data amount is less than the first data amount;

sending m read commands to a non-volatile memory express (NVMe) hard disk, wherein m≤K, and wherein a product of m and the second data amount is not greater than the size of the data buffer of the communication link; and

suspending sending of a remaining read command to the NVMe hard disk in response to congestion occurring on the communication link.

19. The NOF engine according to claim 18 , wherein the program further includes instructions for performing, after the suspending the sending of the remaining read command to the NVMe hard disk:

receiving data read by the NVMe hard disk according to the m read commands;

storing the data in the data buffer corresponding to the communication link; and

sending, in response to the congestion on the communication link being released, the data in the data buffer to the host using the communication link.

20. The NOF engine according to claim 19 , wherein the program further includes performing, after the sending the data in the data buffer to the host:

continuing, by the NOF engine, to send n read commands to the NVMe hard disk, wherein n≤K−m, and wherein a product of n and the second data amount is not greater than the size of the data buffer corresponding to the communication link.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2020
From: ZHANG, SHENGGUANG; HU, HAIYAN; ZHENG, XINQIAN
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 053314/0657 →
Priority Claims (1)
CN 201711444963.0 · Dec 27, 2017 · national
Continuity (2)
Continuation PCTCN2018123749 · Dec 26, 2018
Related Publication 20200326879A1 · Oct 15, 2020
Cited By (5)
US 12,499,040 US 12,541,451 US 12,619,380 US 12,645,397 US 12,711,057