IP Library Granted Patent US 11,210,001
Granted Patent B2
US 11,210,001 · App. 16/855,549 · Granted Dec 28, 2021

Storage device parameter monitoring for load balancing

Inventors: Niles Yang (Mountain View, CA); Phil Reusswig (Mountain View, CA); Sahil Sharma (San Jose, CA); Rohit Sehgal (San Jose, CA)
Assignee: Western Digital Technologies, Inc.
G06F3/0635G06F3/064G06F3/067G06F3/0619G11C16/349
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,210,001
App. No.
16/855,549
Granted
Dec 28, 2021
Kind
B2
Abstract

Systems and methods for storage systems using storage device monitoring for load balancing are described. Storage devices may be configured for data access through a common data stream, such as the storage devices in a storage node or server. Data operations from the common data stream may be distributed among the storage devices using a load balancing algorithm. Performance parameter values, such as grown bad blocks, program-erase cycles, and temperature, may be received for the storage devices and used to determine variance values for each storage device. Variance values demonstrating degrading storage devices may be used to reduce the load allocation of data operations to the degrading storage devices.

Claims (94)

1. A storage system, comprising:

a storage system manager configured to:

manage a plurality of storage devices configured for data access through a common data stream; and

distribute data operations from the common data stream to the plurality of storage devices;

a storage device monitor configured to:

receive, for each storage device of the plurality of storage devices, a grown bad block value;

determine, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;

determine, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;

receive an overprovisioning value for the degrading storage device; and

determine, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold; and

a load balancer configured to:

determine, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;

reduce, responsive to the determination of the degrading storage device, a load allocation of data operations distributed to the degrading storage device;

allocate, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and

skip the degrading storage device for distribution of write data operations from the common data stream.

2. The storage system of claim 1 , wherein:

the load balancing algorithm is configured to distribute data operations evenly across the plurality of storage devices by selecting target storage devices for data operations based on load allocation values for each storage device of the plurality of storage devices; and

reducing the load allocation of data operations distributed to the degrading storage device is based on changing a load allocation value for the degrading storage device in the load balancing algorithm.

3. The storage system of claim 1 , wherein the storage device monitor is further configured to:

receive, for each storage device of the plurality of storage devices, a program erase cycle value;

determine, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices; and

determine, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices.

4. The storage system of claim 3 , wherein the storage system manager is further configured to initiate, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

5. The storage system of claim 1 , wherein the storage device monitor is further configured to:

receive, for each storage device of the plurality of storage devices, a temperature value;

determine, for each storage device of the plurality of storage devices, a third variance value based on temperature values from the plurality of storage devices; and

determine, based on the third variance value, that the degrading storage device has increased temperature relative to the plurality of storage devices.

6. The storage system of claim 5 , wherein the storage system manager is further configured to initiate, responsive to the degrading storage device having increased temperature, an environmental change notification to a cooling system associated with the plurality of storage devices.

7. The storage system of claim 1 , wherein the load balancer is further configured to:

identify a set of data blocks having associated data operations above a hot data threshold; and

allocate the set of data blocks across each of the plurality of storage devices, excluding the degrading storage device.

8. The storage system of claim 7 , wherein:

the load balancer is further configured to rotate the set of data blocks from an initial storage device to the plurality of storage devices, excluding the degrading storage device; and

the load balancer selected the initial storage device to receive the associated data operations for the set of data blocks.

9. The storage system of claim 1 , further comprising:

a storage device performance parameter table configured to store, for each storage device of the plurality of storage devices, performance parameters received from the plurality of storage devices, wherein:

the storage device monitor is further configured to determine the first variance value based on:

aggregating each grown bad block value in successive table entries for each storage device of the plurality of storage devices; and

comparing each grown bad block value to an aggregate value to determine the first variance value for each storage device of the plurality of storage devices; and

the performance parameters include at least one of:

grown bad block values;

program erase cycle values;

temperature values; and

data operations values.

10. A computer-implemented method, comprising:

managing a plurality of storage devices configured for data access through a common data stream;

distributing data operations from the common data stream to the plurality of storage devices;

determining, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;

receiving, for each storage device of the plurality of storage devices, a grown bad block value;

receiving, for each storage device of the plurality of storage devices, a temperature value;

determining, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;

determining, for each storage device of the plurality of storage devices, a third variance value based on temperature values from the plurality of storage devices;

determining, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;

determining, based on the third variance value, that the degrading storage device has increased temperature relative to the plurality of storage devices; and

reducing, responsive to determining that the degrading storage device has increased temperature relative to the plurality of storage devices, a load allocation of data operations distributed to the degrading storage device.

11. The computer-implemented method of claim 10 , further comprising:

receiving an overprovisioning value for the degrading storage device;

determining, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold;

allocating, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and

skipping, based on the secondary storage status, the degrading storage device for distribution of write data operations from the common data stream.

12. The computer-implemented method of claim 10 , wherein:

determining target storage devices includes distributing data operations evenly across the plurality of storage devices by selecting target storage devices for data operations based on load allocation values for each storage device of the plurality of storage devices; and

reducing the load allocation of data operations distributed to the degrading storage device is based on changing a load allocation value for the degrading storage device in the load balancing algorithm.

13. The computer-implemented method of claim 10 , further comprising:

receiving, for each storage device of the plurality of storage devices, a program erase cycle value;

determining, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices; and

determining, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices.

14. The computer-implemented method of claim 13 , further comprising:

initiating, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

15. The computer-implemented method of claim 10 , further comprising:

initiating, responsive to the degrading storage device having increased temperature, an environmental change notification to a cooling system associated with the plurality of storage devices.

16. The computer-implemented method of claim 10 , further comprising:

identifying a set of data blocks having associated data operations above a hot data threshold; and

allocating the set of data blocks across each of the plurality of storage devices, excluding the degrading storage device.

17. The computer-implemented method of claim 16 , further comprising:

rotating, from an initial storage device among the plurality of storage devices, the set of data blocks to the plurality of storage devices, excluding the degrading storage device, wherein the initial storage device received the associated data operations for the set of data blocks.

18. A system, comprising:

a plurality of storage devices configured for data access through a common data stream;

means for distributing data operations from the common data stream to the plurality of storage devices;

means for determining, using a load balancing algorithm, target storage devices among the plurality of storage devices to receive data operations from the common data stream;

means for receiving, for each storage device of the plurality of storage devices, a grown bad block value;

means for receiving, for each storage device of the plurality of storage devices, a program erase cycle value;

means for determining, for each storage device of the plurality of storage devices, a first variance value based on grown bad block values from the plurality of storage devices;

means for determining, for each storage device of the plurality of storage devices, a second variance value based on program erase cycle values from the plurality of storage devices;

means for determining, based on the first variance value, a degrading storage device that has increased grown bad blocks relative to the plurality of storage devices;

means for determining, based on the second variance value, that the degrading storage device has increased program erase cycles relative to the plurality of storage devices; and

means for reducing, responsive to determining that the degrading storage device has increased program erase cycles relative to the plurality of storage devices, a load allocation of data operations distributed to the degrading storage device.

19. The system of claim 18 , further comprising:

means for initiating, responsive to the degrading storage device having increased program erase cycles, a reconditioning process to rewrite data blocks stored on the degrading storage device to the degrading storage device.

20. The system of claim 18 , further comprising:

means for receiving an overprovisioning value for the degrading storage device;

means for determining, based on the overprovisioning value, that the degrading storage device has remaining overprovisioning space below an overprovisioning response threshold;

means for allocating, responsive to the remaining overprovisioning space being below the overprovisioning response threshold, the degrading storage device to a secondary storage status; and

means for skipping, based on the secondary storage status, the degrading storage device for distribution of write data operations from the common data stream.

Assignments (9)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2025
From: SANDISK TECHNOLOGIES, INC.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 070776/0307 →
PATENT COLLATERAL AGREEMENT Recorded Aug 23, 2024
From: SANDISK TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A., AS THE AGENT
Reel/Frame 068762/0494 →
CHANGE OF NAME Recorded Jun 27, 2024
From: SANDISK TECHNOLOGIES, INC.
To: SANDISK TECHNOLOGIES, INC.
Reel/Frame 067982/0032 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2024
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: SANDISK TECHNOLOGIES, INC.
Reel/Frame 067567/0682 →
PATENT COLLATERAL AGREEMENT - A&R LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 064715/0001 →
PATENT COLLATERAL AGREEMENT - DDTL LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 067045/0156 →
RELEASE OF SECURITY INTEREST AT REEL 053482 FRAME 0453 Recorded Feb 8, 2022
From: JPMORGAN CHASE BANK, N.A.
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 058966/0279 →
SECURITY INTEREST Recorded May 14, 2020
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A., AS AGENT
Reel/Frame 053482/0453 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2020
From: YANG, NILES; REUSSWIG, PHIL; SHARMA, SAHIL; SEHGAL, ROHIT
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 052469/0372 →