Granted Patent B2

US 11,593,028 · App. 17/198,551 · Granted Feb 28, 2023

Inventors: Vladimir Shveidel (Pardes-Hana, IL); Alexei Kabishcer (Ramat Gan, IL)

Assignee: EMC IP Holding Company LLC

G06F3/0659G06F3/0613G06F3/0641G06F3/0653G06F3/0673G06N20/00

Abstract

Claims (64)

1. A method of operating a computing device for processing data, the method comprising:

monitoring a set of performance characteristics of the processing of the data, the processing includes (1) receiving data to be written to a data storage system, (2) converting the data to a format for being written to the data storage system, and (3) storing the converted data to persistent storage of the data storage system, the performance characteristics including respective rates of processing storage commands;

periodically calculating, using a predefined set of coefficients, a linear combination of the monitored set of performance characteristics to yield a combined metric, the coefficients being effective to generate the combined metric equal to a normalized value across different hardware platforms at saturation in which the respective hardware platforms are saturated with processing of storage commands; and

upon detecting that the combined metric exceeds a threshold while operating in a first processing mode, transitioning from operating in the first processing mode to operating in a second processing mode, wherein:

the second processing mode has a higher bandwidth than the first processing mode and

processing of data in the second processing mode is less robust than processing of data in the first processing mode,

wherein converting the data to the format for being written to the data storage system includes:

while operating in the first processing mode, performing a deduplication step on the data prior to storing the converted data to the persistent storage, the deduplication step being configured to replace duplicate blocks within the data with pointers to previously-stored versions of those duplicate blocks; and

while operating in the second processing mode, refraining from performing the deduplication step on the data prior to storing the converted data to the persistent storage.

2. The method of claim 1 wherein the monitored set of performance characteristics include:

write operations per unit time;

read operations per unit time;

write bandwidth per unit time; and

read bandwidth per unit time.

3. The method of claim 2 wherein monitoring the set of performance characteristics includes normalizing the monitored set of performance characteristics to processor frequency.

4. The method of claim 1 wherein the method further comprises, upon returning to the first processing mode, performing, as a background operation, the deduplication step on the converted data that was stored to the persistent storage during operation of the second processing mode.

5. The method of claim 1 ,

wherein the computing device has a set of processing cores;

wherein operating in the first processing mode includes:

operating a first subset of the set of processing cores for receiving the data to be written to the data storage system and performing initial processing of the data and

operating a second subset of the set of processing cores for performing deduplication and storing the converted data to persistent storage, the second subset not overlapping with the first subset; and

wherein operating in the second processing mode includes:

operating a third subset of the set of processing cores for receiving the data to be written to the data storage system and performing initial processing of the data, the third subset being larger than the first subset, and

operating a fourth subset of the set of processing cores for storing the converted data to persistent storage, the fourth subset not overlapping with the third subset, the fourth subset being smaller than the second subset.

6. The method of claim 1 wherein the method further comprises, upon detecting that a free amount of persistent storage is below a free space threshold while operating in the second processing mode, transitioning from operating in the second processing mode to operating in the first processing mode.

7. The method of claim 1 wherein the method further comprises, upon detecting that another threshold exceeds the combined metric while operating in the second processing mode, transitioning from operating in the second processing mode to operating in the first processing mode.

8. The method of claim 7 wherein the method further comprises, while operating in the second processing mode, dynamically calculating the other threshold by:

detecting that the combined metric is oscillating within an oscillation envelope;

calculating a difference between the threshold and a width of the oscillation envelope; and

setting the other threshold to be whichever of a default threshold and the calculated difference is lower.

9. The method of claim 1 wherein periodically calculating the combined metric is performed at a periodicity within a range of 1 second to 20 seconds.

10. A computer program product comprising a non-transitory computer-readable storage medium that stores a set of instructions, which, when performed by a computing device, processes data by:

monitoring a set of performance characteristics of the processing of the data, the processing includes (1) receiving data to be written to a data storage system, (2) converting the data to a format for being written to the data storage system, and (3) storing the converted data to persistent storage of the data storage system, the performance characteristics including respective rates of processing storage commands;

periodically calculating, using a predefined set of coefficients, a linear combination of the monitored set of performance characteristics to yield a combined metric, the coefficients being effective to generate the combined metric equal to a normalized value across different hardware platforms at saturation in which the respective hardware platforms are saturated with processing of storage commands; and

upon detecting that the combined metric exceeds a threshold while operating in a first processing mode, transitioning from operating in the first processing mode to operating in a second processing mode, wherein:

the second processing mode has a higher bandwidth than the first processing mode and

processing of data in the second processing mode is less robust than processing of data in the first processing mode,

wherein converting the data to the format for being written to the data storage system includes:

while operating in the first processing mode, performing a deduplication step on the data prior to storing the converted data to the persistent storage, the deduplication step being configured to replace duplicate blocks within the data with pointers to previously-stored versions of those duplicate blocks; and

while operating in the second processing mode, refraining from performing the deduplication step on the data prior to storing the converted data to the persistent storage.

11. The computer program product of claim 10 ,

wherein the computing device has a set of processing cores;

wherein operating in the first processing mode includes:

operating a first subset of the set of processing cores for receiving the data to be written to the data storage system and performing initial processing of the data and

operating a second subset of the set of processing cores for performing deduplication and storing the converted data to persistent storage, the second subset not overlapping with the first subset; and

wherein operating in the second processing mode includes:

operating a third subset of the set of processing cores for receiving the data to be written to the data storage system and performing initial processing of the data, the third subset being larger than the first subset, and

operating a fourth subset of the set of processing cores for storing the converted data to persistent storage, the fourth subset not overlapping with the third subset, the fourth subset being smaller than the second subset.

12. The computer program product of claim 10 wherein the set of instructions, when performed by the computing device, further cause the computing device to, upon detecting that another threshold exceeds the combined metric while operating in the second processing mode, transition from operating in the second processing mode to operating in the first processing mode.

13. The computer program product of claim 12 wherein the set of instructions, when performed by the computing device, further cause the computing device to, while operating in the second processing mode, dynamically calculate the other threshold by:

detecting that the combined metric is oscillating within an oscillation envelope;

calculating a difference between the threshold and a width of the oscillation envelope; and

setting the other threshold to be whichever of a default threshold and the calculated difference is lower.

14. An apparatus comprising:

memory; and

processing circuitry coupled to the memory, configured to processes data by:

monitoring a set of performance characteristics of the processing of the data, the processing includes (1) receiving data to be written to a data storage system, (2) converting the data to a format for being written to the data storage system, and (3) storing the converted data to persistent storage of the data storage system, the performance characteristics including respective rates of processing storage commands;

periodically calculating, using a predefined set of coefficients, a linear combination of the monitored set of performance characteristics to yield a combined metric, the coefficients being effective to generate the combined metric equal to a normalized value across different hardware platforms at saturation in which the respective hardware platforms are saturated with processing of storage commands; and

upon detecting that the combined metric exceeds a threshold while operating in a first processing mode, transitioning from operating in the first processing mode to operating in a second processing mode, wherein:

the second processing mode has a higher bandwidth than the first processing mode and

processing of data in the second processing mode is less robust than processing of data in the first processing mode,

wherein converting the data to the format for being written to the data storage system includes:

while operating in the first processing mode, performing a deduplication step on the data prior to storing the converted data to the persistent storage, the deduplication step being configured to replace duplicate blocks within the data with pointers to previously-stored versions of those duplicate blocks; and

while operating in the second processing mode, refraining from performing the deduplication step on the data prior to storing the converted data to the persistent storage.

Assignments (10)

Continuity (1)