IP Library Granted Patent US 11,507,286
Granted Patent B2
US 11,507,286 · App. 17/323,268 · Granted Nov 22, 2022

Performing storage provision operations on a file system

Inventors: Sicong Yao (Shanghai, CN); Liang Huang (Shanghai, CN); Ruipeng Yang (Shanghai, CN); Jianhua Shao (Shanghai, CN); Xianlong Liu (Shanghai, CN)
Assignee: EMC IP Holding Company LLC
G06F3/0631G06F3/0604G06F3/0659G06F3/0673G06F16/11G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,507,286
App. No.
17/323,268
Granted
Nov 22, 2022
Kind
B2
Abstract

A storage management technique involves: determining a spare degree of physical storage space of a file system and access characteristics of the file system; determining, using a provision operation classification model and based on the spare degree and the access characteristics, a target storage provision operation to be performed for the file system from multiple storage provision operations, wherein the multiple storage provision operations include at least a storage space expansion operation and a storage space reclamation operation, and the provision operation classification model characterizes an association relationship between different spare degrees and different access characteristics of the file system and the multiple storage provision operations; and performing the determined target storage provision operation for the file system. Accordingly, a better balance is achieved between storage efficiency and I/O performance.

Claims (110)

1. A method for storage management, comprising:

determining a spare degree of physical storage space of a file system and access characteristics of the file system;

determining, using a provision operation classification model and based on the spare degree and the access characteristics, a target storage provision operation to be performed for the file system from multiple storage provision operations, wherein the multiple storage provision operations include at least a storage space expansion operation and a storage space reclamation operation, and the provision operation classification model characterizes an association relationship between different spare degrees and different access characteristics of the file system and the multiple storage provision operations; and

performing the determined target storage provision operation for the file system;

acquiring a training spare degree and training access characteristics of the file system at a particular time point;

determining a classification label corresponding to the training spare degree and the training access characteristics, wherein the classification label indicates that one of the multiple storage provision operations is to be performed on the file system; and

training the provision operation classification model based on the training spare degree, the training access characteristics, and the classification label.

2. The method according to claim 1 , wherein determining the access characteristics includes determining at least one of the following:

a sequential write capability of the file system; and

a predicted degree of consumption of spare storage space in access operations to be performed for the file system.

3. The method according to claim 2 , wherein determining the sequential write capability includes:

determining a first number of storage units in the physical storage space of the file system that are used for sequential writing;

determining a second number of storage units in which data is partially written among the storage units of the file system that are used for sequential writing; and

determining the sequential write capability based on a ratio of the second number to the first number.

4. The method according to claim 2 , wherein determining a predicted degree of consumption of spare storage space includes:

determining a third number of access operations to be performed;

determining a fourth number of pending access operations to which spare storage space is to be allocated for writing data; and

determining the predicted degree of consumption of spare storage space based on a ratio of the fourth number to the third number.

5. The method according to claim 1 , wherein determining a target storage provision operation to be performed includes:

mapping the spare degree and the access characteristics to a target score within a score range using the provision operation classification model;

determining that the storage space expansion operation is to be performed for the file system if the target score falls within a first score interval associated with the storage space expansion operation in the score range; and

determining that the storage space reclamation operation is to be performed for the file system if the target score falls within a second score interval associated with the storage space reclamation operation in the score range.

6. The method according to claim 5 , wherein the score range further includes a gap interval not within the first score interval and the second score interval, and the multiple storage provision operations further include a provision maintenance operation, and wherein determining a target storage provision operation to be performed further includes:

determining, if it is determined that the target score is in the gap interval, that the provision maintenance operation is to be performed for the file system to maintain provision of the physical storage space of the file system.

7. The method according to claim 5 , wherein the provision operation classification model is based on a function having a cumulative distribution characteristic.

8. The method according to claim 1 , wherein determining the classification label includes:

determining, within a time period after the particular time point, a first size of storage space that is allocated for storing data and a second size of storage space that is released in the storage space of the file system; and

determining the classification label based on the first size of storage space and the second size of storage space.

9. The method according to claim 8 , wherein the provision operation classification model maps the spare degree and the access characteristics to a target score within a score range, the score range including a first score interval associated with the storage space expansion operation and a second score interval associated with the storage space reclamation operation, and determining the classification label includes:

determining the classification label as a first score within the first score interval if a ratio of the first size of storage space to the second size of storage space is greater than a first threshold associated with the storage space expansion operation; and

determining the classification label as a second score within the second score interval if the ratio of the first size of storage space to the second size of storage space is less than a second threshold associated with the storage space reclamation operation.

10. The method according to claim 9 , wherein the score range further includes a gap interval not within the first score interval and the second score interval, and determining the classification label further includes:

determining the classification label as a third score within the gap interval if the first size of storage space and the second size of storage space are both zero;

determining the classification label as a fourth score within the second score interval if the first size of storage space is zero; and

determining the classification label as a fifth score within the first score interval if the second size of storage space is zero.

11. The method according to claim 9 , wherein the physical storage space of the file system is not expanded or reclaimed during the time period.

12. The method according to claim 1 , further comprising:

determining at least one of an error rate of the provision operation classification model and a utilization per spare storage space of the file system after the target storage provision operation is performed; and

determining re-training of the provision operation classification model based on the determined at least one of the error rate and the utilization per spare storage space.

13. The method according to claim 12 , wherein determining the utilization per spare storage space includes:

determining an increase amount of spare storage space in the file system over a time period;

determining an access capacity of the file system over the time period; and

determining the utilization per spare storage space based on a ratio of the access amount to the increase amount.

14. The method according to claim 12 , wherein determining re- training of the provision operation classification model includes:

determining that re-training is to be performed on the provision operation classification model if at least one of the following is determined:

the error rate exceeds a threshold error rate, or

the utilization per spare storage space has a downtrend relative to a previous utilization per spare storage space of the file system.

15. The method according to claim 1 , further comprising:

averaging multiple classification labels corresponding to different time points, including the determined classification label, to obtain an averaged classification label;

wherein training the provision operation classification model is further based on the averaged classification model.

16. The method according to claim 1 , wherein the classification model indicates a target result corresponding to the training spare degree and the training access characteristics;

wherein training the provision operation classification model includes:

generating an output classification label using the provision operation classification model and corresponding to the training spare degree and the training access characteristics;

calculating an error amount based on the target result and the output classification label;

adjusting parameters of the provision operation classification model based on the error amount.

17. An electronic device, comprising:

at least one processor; and

at least one memory storing computer program instructions, wherein the at least one memory and the computer program instructions are configured to cause, together with the at least one processor, the electronic device to perform actions including:

determining a spare degree of physical storage space of a file system and access characteristics of the file system;

determining, using a provision operation classification model and based on the spare degree and the access characteristics, a target storage provision operation to be performed for the file system from multiple storage provision operations, wherein the multiple storage provision operations include at least a storage space expansion operation and a storage space reclamation operation, and the provision operation classification model characterizes an association relationship between different spare degrees and different access characteristics of the file system and the multiple storage provision operations; and

performing the determined target storage provision operation for the file system;

acquiring a training spare degree and training access characteristics of the file system at a particular time point;

determining a classification label corresponding to the training spare degree and the training access characteristics, wherein the classification label indicates that one of the multiple storage provision operations is to be performed on the file system; and

training the provision operation classification model based on the training spare degree, the training access characteristics, and the classification label.

18. The device according to claim 17 , wherein determining the access characteristics includes determining at least one of the following:

a sequential write capability of the file system; and

a predicted degree of consumption of spare storage space in access operations to be performed for the file system.

19. The device according to claim 18 , wherein determining the sequential write capability includes:

determining a first number of storage units in the physical storage space of the file system that are used for sequential writing;

determining a second number of storage units in which data is partially written among the storage units of the file system that are used for sequential writing; and

determining the sequential write capability based on a ratio of the second number to the first number.

20. The device according to claim 18 , wherein determining a predicted degree of consumption of spare storage space includes:

determining a third number of access operations to be performed;

determining a fourth number of pending access operations to which spare storage space is to be allocated for writing data; and

determining the predicted degree of consumption of spare storage space based on a ratio of the fourth number to the third number.

21. The device according to claim 17 , wherein determining a target storage provision operation to be performed includes:

mapping the spare degree and the access characteristics to a target score within a score range using the provision operation classification model;

determining that the storage space expansion operation is to be performed for the file system if the target score falls within a first score interval associated with the storage space expansion operation in the score range; and

determining that the storage space reclamation operation is to be performed for the file system if the target score falls within a second score interval associated with the storage space reclamation operation in the score range.

22. The device according to claim 21 , wherein the score range further includes a gap interval not within the first score interval and the second score interval, and the multiple storage provision operations further include a provision maintenance operation, and wherein determining a target storage provision operation to be performed further includes:

determining, if it is determined that the target score is in the gap interval, that the provision maintenance operation is to be performed for the file system to maintain provision of the physical storage space of the file system.

23. The device according to claim 21 , wherein the provision operation classification model is based on a function having a cumulative distribution characteristic.

24. The device according to claim 17 , wherein determining the classification label includes:

determining, within a time period after the particular time point, a first size of storage space that is allocated for storing data and a second size of storage space that is released in the storage space of the file system; and

determining the classification label based on the first size of storage space and the second size of storage space.

25. The device according to claim 24 , wherein the provision operation classification model maps the spare degree and the access characteristics to a target score within a score range, the score range including a first score interval associated with the storage space expansion operation and a second score interval associated with the storage space reclamation operation, and determining the classification label includes:

determining the classification label as a first score within the first score interval if a ratio of the first size of storage space to the second size of storage space is greater than a first threshold associated with the storage space expansion operation; and

determining the classification label as a second score within the second score interval if the ratio of the first size of storage space to the second size of storage space is less than a second threshold associated with the storage space reclamation operation.

26. The device according to claim 25 , wherein the score range further includes a gap interval not within the first score interval and the second score interval, and determining the classification label further includes:

determining the classification label as a third score within the gap interval if the first size of storage space and the second size of storage space are both zero;

determining the classification label as a fourth score within the second score interval if the first size of storage space is zero; and

determining the classification label as a fifth score within the first score interval if the second size of storage space is zero.

27. The device according to claim 25 , wherein the physical storage space of the file system is not expanded or reclaimed during the time period.

28. The device according to claim 17 , wherein the actions further include:

determining at least one of an error rate of the provision operation classification model and a utilization per spare storage space of the file system after the target storage provision operation is performed; and

determining re-training of the provision operation classification model based on the determined at least one of the error rate and the utilization.

29. The device according to claim 28 , wherein determining the utilization per spare storage space includes:

determining an increase amount of spare storage space in the file system over a time period;

determining an access capacity of the file system over the time period; and

determining the utilization per spare storage space based on a ratio of the access amount to the increase amount.

30. The device according to claim 28 , wherein determining re- training of the provision operation classification model includes:

determining that re-training is to be performed on the provision operation classification model if at least one of the following is determined:

the error rate exceeds a threshold error rate, or the utilization per spare storage space has a downtrend relative to a previous utilization per spare storage space of the file system.

31. A computer program product having a non-transitory computer readable medium which stores a set of instructions to perform storage management; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:

determining a spare degree of physical storage space of a file system and access characteristics of the file system;

determining, using a provision operation classification model and based on the spare degree and the access characteristics, a target storage provision operation to be performed for the file system from multiple storage provision operations, wherein the multiple storage provision operations include at least a storage space expansion operation and a storage space reclamation operation, and the provision operation classification model characterizes an association relationship between different spare degrees and different access characteristics of the file system and the multiple storage provision operations; and

performing the determined target storage provision operation for the file system;

acquiring a training spare degree and training access characteristics of the file system at a particular time point;

determining a classification label corresponding to the training spare degree and the training access characteristics, wherein the classification label indicates that one of the multiple storage provision operations is to be performed on the file system; and

training the provision operation classification model based on the training spare degree, the training access characteristics, and the classification label.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (058014/0560) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 062022/0473 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (057931/0392) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 062022/0382 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (057758/0286) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 061654/0064 →
SECURITY INTEREST Recorded Oct 6, 2021
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 058014/0560 →
SECURITY INTEREST Recorded Oct 6, 2021
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 057758/0286 →
SECURITY INTEREST Recorded Oct 6, 2021
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 057931/0392 →
SECURITY AGREEMENT Recorded Oct 1, 2021
From: DELL PRODUCTS, L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 057682/0830 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2021
From: YAO, SICONG; HUANG, LIANG; YANG, RUIPENG; SHAO, JIANHUA; LIU, XIANLONG
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 056438/0070 →
Priority Claims (1)
CN 202110089226.3 · Jan 22, 2021 · national
Continuity (1)
Related Publication 20220236897A1 · Jul 28, 2022