IP Library Granted Patent US 9,927,983
Granted Patent B2
US 9,927,983 · App. 14/868,373 · Granted Mar 27, 2018

Methods, systems and computer readable media for intelligent fetching of data storage device commands from submission queues

Inventors: Shay Benisty (Beer Sheva, IL); Noga Harari Shechter (Lehavim, IL); Amir Segev (Meitar, IL); Tal Sharifie (Lehavim, IL)
Assignee: SanDisk Technologies LLC
G06F3/061G06F3/0653G06F3/0659G06F3/0688
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,927,983
App. No.
14/868,373
Granted
Mar 27, 2018
Kind
B2
Abstract

Methods, systems, and computer readable media for intelligent fetching of storage device commands from submission queues are disclosed. On method is implemented in a data storage device including a controller and a memory. The method includes collecting submission queue command statistics; monitoring resource state of the data storage device. The method further includes using the submission queue command statistics and the resource state to select a submission queue from which a next data storage device command should be fetched. The method further includes fetching the command from the selected submission queue. The method further includes providing the command to command processing logic.

Claims (63)

1. A method, comprising:

collecting command statistics pertaining to storage device commands previously fetched from respective submission queues of a plurality of submission queues associated with a data storage device;

fetching storage device commands from the submission queues, wherein fetching a storage device command comprises:

selecting a submission queue from which to fetch the storage device command based on the command statistics pertaining to the storage device commands previously fetched from the respective submission queues and a state of one or more resources of the data storage device;

fetching the command from the selected submission queue; and

providing the command to command processing logic,

wherein collecting the command statistics comprises collecting information regarding whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands.

2. The method of claim 1 , wherein collecting the command statistics further comprises collecting statistics regarding a number of pending storage device commands in the respective submission queues.

3. The method of claim 1 , wherein collecting the command statistics further comprises collecting an indication of a number of storage device commands previously fetched from the respective submission queues.

4. The method of claim 1 , wherein collecting the command statistics further comprises collecting command type information for the storage device commands previously fetched from the respective submission queues.

5. The method of claim 1 , wherein:

collecting the command statistics comprises determining whether the respective submission queues are likely to comprise one of: read commands and write commands based on the storage device commands previously fetched from the respective submission queues, and

selecting the submission queue from which to fetch the storage device command from a submission queue determined to comprise one of: read commands and write commands.

6. The method of claim 1 , further comprising determining the state of the one or more resources of the data storage device.

7. The method of claim 6 , wherein determining the state of the one or more resources of the data storage device comprises monitoring one or more of:

a read pipeline of the data storage device,

a write pipeline of the data storage device, and

a memory page being filled by the data storage device.

8. The method of claim 1 , wherein selecting the submission queue further comprises:

determining one or more characteristics of storage commands likely queued within the respective submission queues based on the command statistics pertaining to the storage device commands previously fetched from the respective submission queues, and

comparing the determined characteristics to the state of the one or more resources of the data storage device.

9. The method of claim 1 , wherein selecting the submission queue further comprises:

using the command statistics pertaining to the storage device commands previously fetched from the respective submission queues to identify submission queues likely to comprise one of: read commands and write commands;

selecting a submission queue identified as being likely to comprise read commands in response to determining that the data storage device is currently unable to process write commands based on the state of the one or more resources of the data storage device; and

selecting a submission queue identified as being likely to comprise write commands in response to determining that the data storage device is currently unable to process read commands based on the state of the one or more resources of the data storage device.

10. The method of claim 1 , wherein collecting the command statistics comprises continually collecting the command statistics pertaining to storage device commands being fetched from the respective submission queues.

11. The method of claim 1 , further comprising:

initially operating in a static mode for selecting the submission queues from which to fetch storage device commands; and

switching to using the command statistics for selecting the submission queues from which to fetch storage device commands in response to a predetermined trigger.

12. The method of claim 1 , wherein the data storage device comprises a nonvolatile memory device.

13. The method of claim 12 , wherein the data storage device comprises a nonvolatile memory controller.

14. The method of claim 1 , wherein the memory comprises one or more of: two dimensional NAND flash memory, and three dimensional NAND flash memory.

15. A data storage device, comprising:

a memory;

a resource monitor for monitoring a resource state of the data storage device;

a command monitor for collecting statistics pertaining to device storage commands previously fetched from respective submission queues of a plurality of submission queues, wherein collecting the command statistics comprises collecting information regarding whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands; and

a command fetcher for fetching storage device commands from the submission queues and providing the fetched storage device commands to command processing logic, wherein fetching a storage device command comprises:

selecting one of the plurality of submission queues from which to fetch the storage device command based on the resource state of the data storage device and the collected statistics pertaining to the device storage commands previously fetched from the respective submission queues, and

fetching the storage device command from the selected submission queue.

16. The data storage device of claim 15 , wherein the command monitor is configured to collect statistics regarding one or more of a type of the storage device commands previously fetched from the respective submission queues and a size of the storage device commands previously fetched from the respective submission queues.

17. The data storage device of claim 15 , wherein the command monitor is further configured to collect an indication of a number of storage device commands previously fetched from the respective submission queues.

18. The data storage device of claim 15 , wherein the command monitor is configured to collect statistics regarding a ratio of read commands to write commands of the storage device commands previously fetched from the respective submission queues.

19. The data storage device of claim 15 , wherein the collected statistics identify submission queues that have historically contained a majority of one of: read commands and write commands.

20. The data storage device of claim 15 , wherein the resource monitor is configured to monitor one or more of: a read pipeline state of the data storage device, a write pipeline state of the data storage device, and a page being filled by the data storage device.

21. The data storage device of claim 15 , wherein the collected statistics identify submission queues likely to contain read commands and submission queues likely to contain write commands based on the storage device commands previously fetched from the respective submission queues.

22. The data storage device of claim 21 wherein in response to the resource state of the data storage device indicating that a read pipeline of the data storage device is currently unavailable to process read commands, the command fetcher is configured to select the submission queue from the submission queues identified as being likely to contain write commands, bypassing selection from the submission queues identified as being likely to contain read commands.

23. The data storage device of claim 21 , wherein in response to the resource state of the data storage device indicating that a write pipeline of the data storage device is currently unavailable to process write commands, the command fetcher is configured to select the submission queue from the submission queues identified as being likely to contain read commands, bypassing selection from the submission queues identified as being likely to contain write commands.

24. The data storage device of claim 15 , wherein:

the collected statistics indicate submission queues, of the plurality of submission queues, that are likely to contain one of: random input/output (I/O) commands and sequential I/O commands; and

the command fetcher is configured to select the submission queue from the submission queues indicated as being likely to contain random I/O commands, bypassing selection from the submission queues indicated as being likely to contain sequential I/O commands, in response to the monitored resource state indicating that the data storage device is filling a memory page with data corresponding to random I/O commands.

25. The data storage device of claim 15 , wherein the command fetcher is configured to switch from a static submission queue selection mode to a dynamic selection mode based on the resource state of the data storage device and the collected statistics pertaining to the storage device commands previously fetched from the respective submission queues in response to a predetermined trigger.

26. The data storage device of claim 15 , further comprising a controller, wherein the command monitor, the resource monitor, and the command fetcher are components of the controller.

27. The data storage device of claim 15 , wherein the memory comprises a nonvolatile memory, and the data storage device further comprises a memory controller.

28. The data storage device of claim 27 , wherein the memory controller comprises a nonvolatile memory controller.

29. The data storage device of claim 28 , wherein the nonvolatile memory controller comprises a nonvolatile memory express (NVMe) controller.

30. The data storage device of claim 15 , wherein the memory comprises one or more of: two dimensional NAND flash memory and three dimensional NAND flash memory.

31. A non-transitory computer readable medium having stored thereon instructions to configure a processor of a computing device to perform operations, comprising:

collecting submission queue command statistics pertaining to commands previously fetched from respective submission queues of a plurality of submission queues, the command statistics indicating whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands;

monitoring a resource state of a data storage device including a memory and a controller; and

fetching commands from the submission queues, wherein fetching a command comprises:

using the submission queue command statistics pertaining to the commands previously fetched from the respective submission queues and the resource state to select a submission queue from which to fetch the command,

fetching the command from the selected submission queue, and

providing the command to command processing logic.

Assignments (5)
PARTIAL RELEASE OF SECURITY INTERESTS Recorded Apr 25, 2025
From: JPMORGAN CHASE BANK, N.A., AS AGENT
To: SANDISK TECHNOLOGIES, INC.
Reel/Frame 071382/0001 →
SECURITY AGREEMENT Recorded Apr 25, 2025
From: SANDISK TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 071050/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 31, 2024
From: SANDISK TECHNOLOGIES LLC
To: SANDISK TECHNOLOGIES, INC.
Reel/Frame 069796/0423 →
CHANGE OF NAME Recorded May 25, 2016
From: SANDISK TECHNOLOGIES INC
To: SANDISK TECHNOLOGIES LLC
Reel/Frame 038812/0954 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2015
From: BENISTY, SHAY; SHECHTER, NOGA HARARI; SEGEV, AMIR; SHARIFIE, TAL
To: SANDISK TECHNOLOGIES INC.
Reel/Frame 036835/0045 →
Continuity (1)
Related Publication 20170090753A1 · Mar 30, 2017