Method and apparatus for managing storage system
Embodiments of the present disclosure provide methods, apparatuses and computer program products for managing a storage system. The storage system comprises a plurality of cache devices and a bottom storage device, and the plurality of cache devices comprise a first cache device group and a second cache device group. The method according to an aspect of the present disclosure comprises: receiving an input/output (I/O) request for the storage device; in response to determining that the I/O request triggers caching of target data, storing the target data from the storage device into the first cache device group if the I/O request is a read request; and storing the target data into the second cache device group if the I/O request is a write request. Embodiments of the present disclosure introduce a new architecture for cache devices so that the processing delay is shortened, and/or, the storage capacity can be used more effectively.
1. A method of managing a plurality of cache devices, the plurality of cache devices including a first cache device group serving a read request and a second cache device group serving a write request, the method comprising:
detecting a first request received from a user for expanding the plurality of cache devices;
in response to and after detecting the first request from the user, obtaining status information of the plurality of cache devices, the status information indicating at least one of: a load status and a usage situation of a storage space;
adding, based on the obtained status information, a new cache device into the first cache device group or the second cache device group, the new cache device previously being separate from the first cache device group and the second cache device group;
determining a migration direction between the first cache device group and the second cache device group based on values of IOPs (I/O's per second) of the first cache device group and the second cache device group; and
performing migration of a cache device, wherein performing migration of the cache device moves the cache device in the determined migration direction and changes an attribute of the cache device between read-write and read-only.
2. The method according to claim 1 , wherein the first request indicates a list of a plurality of candidate cache devices, and wherein adding the new cache device comprises:
selecting new cache devices one by one from the list of the candidate cache devices for adding the new cache device into the first cache device group or the second cache device group.
3. The method according to claim 1 , wherein the status information indicates the load status of the plurality of cache devices, and adding the new cache device comprises:
obtaining, based on the load status of the plurality of cache device, a first load of the first cache device group and a second load of the second cache device group;
in response to a difference between the first load and the second load or a ratio of the first load to the second load exceeds a predetermined threshold, adding the new cache device into the first cache device group; and
in response to the difference between the first load and the second load or the ratio of the first load to the second load is below the predetermined threshold, adding the new cache device into the second cache device group.
4. The method according to claim 1 , wherein the status information indicates the usage situation of the storage space of the plurality of cache devices, and adding the new cache device comprises:
determining, based on the usage situation of the storage space of the plurality of cache devices, a usage rate of the storage space of the second cache device group;
in response to the usage rate of the storage space of the second cache device group exceeds a first threshold, adding the new cache device into the second cache device group; and
in response to the usage rate of the storage space of the second cache device group is below the first threshold, adding the new cache devices into the first cache device group.
5. The method according to claim 1 , wherein the status information indicates the usage situation of the storage space of the plurality of cache devices, and adding the new cache device comprises:
determining, based on the usage situation of the storage space of the plurality of cache devices, an idle rate of the storage space of the second cache device group;
in response to the idle rate of the storage space of the second cache device group is below a second threshold, adding the new cache device into the second cache device group; and
in response to the idle rate of the storage space of the second cache device group is above the second threshold, adding the new cache device into the first cache device group.
6. The method according to claim 1 , wherein a cache device in the first cache device group comprises a first solid-state disk (SSD) pair, and each SSD in the first SSD pair serves the read request independently.
7. The method according to claim 1 , further comprising:
receiving a balance trigger request for balancing the first cache device group and the second cache device group; and
in response to detecting the balance trigger request, obtaining new status information of the plurality of cache devices,
wherein performing migration of the cache device between the first cache device group and the second cache device group is based on the obtained status information of the plurality of cache devices.
8. The method of claim 1 , further comprising storing a data structure to maintain the attribute of the cache device, the data structure including:
a type flag to indicate the attribute;
at least one disk serial number of the cache device;
an online time indicator of the cache device;
a load status indicator of the cache device; and
a disk space usage indicator of the cache device.
9. A method of managing a plurality of cache devices in a storage system, the plurality of cache devices including a first cache device group serving a read request and a second cache device group serving a write request, the method comprising:
detecting a shrinking request from a user for shrinking the cache devices; and
in response to and after detecting the shrinking request from the user,
removing at least one cache device in the second cache device group from the plurality of cache devices by a removal procedure configured to:
determine whether the at least one cache device is in the first cache device group or the second cache device group;
in response to a determination that the at least one cache device is in the second cache device group,
prohibit a write operation for the at least one cache device;
duplicate a dirty page in the at least one cache device to a storage device of the storage system; and
remove the at least one cache device;
in response to a determination that the at least one cache device is in the first cache device group, remove the at least one cache device without prohibiting a write operation or duplicating a dirty page; and
determine a migration direction for migrating a cache device between the first cache device group and the second cache device group based on values of IOPs (I/O's per second) of the first cache device group and the second cache device group,
performing migration of a cache device, wherein performing migration of the cache device moves the cache device in the determined migration direction and changes an attribute of the cache device between read-write and read-only.
10. The method according to claim 9 , wherein the shrinking request indicates a list of candidate cache devices, and the method further comprises:
selecting cache devices one by one from the list of the candidate cache devices for removing the selected cache devices from the plurality of cache devices.
11. The method according to claim 9 , wherein a cache device in the first cache device group comprises a first solid-state disk (SSD) pair, and each SSD in the first SSD pair serves the read request independently.
12. A storage system, comprising:
a storage device;
a plurality of cache devices including a first cache device group serving a read request and a second cache device group serving a write request; and
a storage system manager configured to execute the method of managing the plurality of cache devices, the method comprising:
detecting a first request received from a user for expanding the plurality of cache devices;
in response to and after detecting the first request from the user, obtaining status information of the plurality of cache devices, the status information indicating at least one of: a load status and a usage situation of a storage space;
adding, based on the obtained status information, a new cache device into the first cache device group or the second cache device group, the new cache device previously being separate from the first cache device group and the second cache device group; and
determining a migration direction between the first cache device group and the second cache device group based on values of IOPs (I/O's per second) of the first cache device group and the second cache device group; and
performing migration of a cache device, wherein performing migration of the cache device moves the cache device in the determined migration direction and changes an attribute of the cache device between read-write and read-only.
13. The storage system of claim 12 , wherein the first request indicates a list of a plurality of candidate cache devices, and wherein adding the new cache device comprises:
selecting new cache devices one by one from the list of the candidate cache devices for adding the new cache device into the first cache device group or the second cache device group.
14. The storage system of claim 12 , wherein the status information indicates the load status of the plurality of cache devices, and adding the new cache device comprises:
obtaining, based on the load status of the plurality of cache device, a first load of the first cache device group and a second load of the second cache device group;
in response to a difference between the first load and the second load or a ratio of the first load to the second load exceeds a predetermined threshold, adding the new cache device into the first cache device group; and
in response to the difference between the first load and the second load or the ratio of the first load to the second load is below the predetermined threshold, adding the new cache device into the second cache device group.
15. The storage system of claim 12 , wherein the status information indicates the usage situation of the storage space of the plurality of cache devices, and adding the new cache device comprises:
determining, based on the usage situation of the storage space of the plurality of cache devices, a usage rate of the storage space of the second cache device group;
in response to the usage rate of the storage space of the second cache device group exceeds a first threshold, adding the new cache device into the second cache device group; and
in response to the usage rate of the storage space of the second cache device group is below the first threshold, adding the new cache devices into the first cache device group.
16. The storage system of claim 12 , wherein the status information indicates the usage situation of the storage space of the plurality of cache devices, and adding the new cache device comprises:
determining, based on the usage situation of the storage space of the plurality of cache devices, an idle rate of the storage space of the second cache device group;
in response to the idle rate of the storage space of the second cache device group is below a second threshold, adding the new cache device into the second cache device group; and
in response to the idle rate of the storage space of the second cache device group is above the second threshold, adding the new cache device into the first cache device group.
17. The storage system of claim 12 , wherein a cache device in the first cache device group comprises a first solid-state disk (SSD) pair, and each SSD in the first SSD pair serves the read request independently.
18. The storage system of claim 12 , wherein the method further comprises:
detecting a second request for shrinking the cache devices; and
in response to detecting the second request,
removing at least one cache device in the second cache device group from the plurality of cache devices by:
prohibiting a write operation for the at least one cache device;
duplicating a dirty page in the at least one cache device to a storage device of the storage system; and
removing the at least one cache device.
19. The storage system of claim 12 , wherein the method further comprises:
detecting a second request for shrinking the cache devices; and
in response to detecting the second request, removing a first cache device in the first cache device group from the plurality of cache devices.