Metadata updating
A storage system and method of operating the storage system is described where the metadata used to access the data stored in a storage device is distributed amongst a plurality of controllers in communication with a user, with each other and with the storage device. Each controller stores at least the metadata needed to access the data relevant to the user at the time, and metadata is updated to respond to internal system activities such as device failures, snapshots, backup operations or the like. To preserve coherence of the metadata, each metadata update is communicated to the other controllers and the storage device. The update is either transmitted to the other controllers and to the storage device and each metadata location is updated, or the update is transmitted to the storage device and each of the controllers is instructed to request an update from the storage device.
1. A data storage system, comprising:
a storage media comprising a non-volatile storage device having a communications interface with a management sub-system; and
the management sub-system comprising a plurality of controllers in communication with the storage media, wherein the plurality of controllers are in communications with the storage device and with an external user,
wherein
the management sub-system maintains metadata in the storage media, the metadata containing relationships between logical block addresses (LBAs) and the location of the LBAs in the storage media and each of the controllers maintains an image of the metadata identical to the metadata images on the other controllers, the metadata image corresponding to at least current active user access requirements,
a controller of the plurality of controllers is assigned as a master controller,
the master controller sends changed metadata either to each of the controllers or to the storage device,
the master controller instructs each relevant controller to read metadata from the storage device to perform an update,
the master controller adds the update to a log file and sends the log file to each of the other controllers and directs each of the controllers having overlapping storage allocation to update the metadata at each location with the log file when the update to the metadata is less than a specific value, and
the master controller sends the update to the storage media to update the metadata and directs each of the controllers to retrieve the updated metadata from the storage media when the update to the metadata is more than the specified value.
2. The system of claim 1 , wherein the changed metadata sent to the storage device modifies an image of the metadata by storing the metadata changes in the log file and a local controller is configured to modify the metadata image using the log file.
3. The system of claim 2 , wherein metadata in the log file is processed to update the image of the metadata and a pointer in the log file establishes a common update point for all of the controllers reading the data.
4. The system of claim 1 , wherein a round-robin token is used to designate a controller as the master controller.
5. The system of claim 1 , wherein the master controller polls each of the other controllers of the plurality of controllers and transfers the designation of master controller to the controller of the plurality of controllers that has a pending metadata update to perform.
6. A method of managing distributed metadata in a storage system, comprising:
providing a plurality of controllers in communication with a common storage medium;
connecting the controllers and the storage medium using a network;
maintaining metadata at each of the controllers containing relationships between logical block addresses (LBAs) or names of files in user requests and the location of the LBAs or files in the common storage medium;
assigning one of the controllers as a master controller;
updating the metadata at each of the controllers having an overlapping storage allocation, and updating the metadata on the common storage medium;
wherein the step of updating includes the master controller:
if the update is less than a specified value, sending a log file including the update to each of the controllers having the overlapping storage allocation and to the metadata on the common storage medium and storing the updates in a file at each location; and
if the update is more than the specified value, sending the update to the common storage location to update the metadata and directing each of the other controllers to retrieve one of:
the log file; or,
the updated metadata from the common storage medium.
7. The method of claim 6 , wherein for metadata update initiated by the master controller of the plurality of controllers includes:
assigning a unique identifying tag value; and
evaluating the tag value to order the updates in time order.
8. The method of claim 7 , wherein using the log file to update the metadata includes:
determining a tag value such that each log file used to update metadata at each controller and the common storage medium is processed beginning at a time determined by the identifying tag value.
9. A computer program product stored on a non-transient computer readable medium, comprising:
instructions for configuring a plurality of controllers and a storage medium controller in mutual communication with each other and a user to perform the steps of:
maintaining metadata at each controller containing relationships between logical block addresses (LBAs) or names of files in user requests and locations of the LBAs or files on the storage medium;
maintaining metadata at the storage medium containing the relationships between the logical block addresses (LBAs) or names of files in user requests and the locations of the LBAs or files on the storage medium, wherein the metadata is accessible by the plurality of controllers;
updating the metadata with a master controller of the plurality of controllers and updating the metadata with the other controllers and the storage medium controller by one of:
the master controller sending the metadata update in a log file to each of the other controllers and the storage medium controller if the update is below a specified value; and
the master controller sending the metadata update to the storage medium controller and instructing the plurality of controllers to retrieve one of an update the log file or the metadata stored in the storage medium controller if the update is above the specified value.
10. A data storage system, comprising:
a storage media comprising a non-volatile storage device having a communications interface with a management sub-system; and
the management sub-system comprising a plurality of controllers in communication with the storage media, wherein the plurality of controllers are in communications with the storage device and with an external user, wherein:
the management sub-system is configured to maintain metadata in the storage media, the metadata containing relationships between logical block addresses (LBAs) and the location of the LBAs in the storage media and each of the controllers is configured to maintain an image of the metadata identical to the metadata images on the other controllers, the metadata image corresponding to at least current active user access requirements,
a controller of the plurality of controllers is assigned as a master controller,
the master controller is configured to send changed metadata either to each of the controllers or to the storage device,
the master controller is configured to instruct each relevant controller to read metadata from the storage device to perform an update,
the master controller is configured to add the update to a log file and send the log file to each of the other controllers and direct each of the controllers having overlapping storage allocation to update the metadata at each location with the log file if the update to the metadata is in a first logical locality, and
the master controller is configured to send the update to the storage media to update the metadata and direct each of the controllers to retrieve the updated metadata from the storage media if the update to the metadata is in a second logical locality.
11. The system of claim 10, wherein the changed metadata sent to the storage device modifies an image of the metadata by storing the metadata changes in the log file and a local controller is configured to modify the metadata image using the log file.
12. The system of claim 11, wherein metadata in the log file is processed to update the image of the metadata and a pointer in the log file establishes a common update point for all of the controllers reading the data.
13. The system of claim 10, wherein a round-robin token is used to designate a controller as the master controller.
14. The system of claim 10, wherein the master controller is configured to poll each of the other controllers of the plurality of controllers and transfer the designation of master controller to the controller of the plurality of controllers that has a pending metadata update to perform.
15. A method of managing distributed metadata in a storage system, comprising:
providing a plurality of controllers in communication with a common storage medium;
connecting the controllers and the storage medium using a network;
maintaining metadata at each of the controllers containing relationships between logical block addresses (LBAs) or names of files in user requests and the location of the LBAs or files in the common storage medium;
assigning one of the controllers as a master controller;
updating the metadata at each of the controllers having an overlapping storage allocation, and
updating the metadata on the common storage medium;
wherein the step of updating comprises the master controller:
if the update is in a first logical locality, sending a log file including the update to each of the controllers having the overlapping storage allocation and to the metadata on the common storage medium and storing the updates in a file at each location; and
if the update is in a second logical locality, sending the update to the common storage location to update the metadata and directing each of the other controllers to retrieve one of: the log file; or, the updated metadata from the common storage medium.
16. The method of claim 15, wherein for metadata update initiated by the master controller of the plurality of controllers comprises:
assigning a unique identifying tag value; and
evaluating the tag value to order the updates in time order.
17. The method of claim 16, wherein using the log file to update the metadata comprises:
determining a tag value such that each log file used to update metadata at each controller and the common storage medium is processed beginning at a time determined by the identifying tag value.