Asynchronous namespace maintenance
Asynchronous namespace maintenance in a distributed replicated data storage system is disclosed. An access device/program serving as a front end to the distributed replicated data storage system updates a batch of updated meta data about stored data items when data items are stored in the distributed replicated data storage system. When the elapsed time since the last batch of data item meta data was stored exceeds a first threshold value or the current batch size exceeds a second threshold value, the access device/program stores the current batch of updated meta data as an object in the distributed replicated data storage system, receiving a batch object identifier for the stored batch of updated meta data, and distributes the batch object identifier to other access devices and/or access programs which retrieve the batch of updated meta data and update their namespaces.
1. A system comprising:
a distributed replicated data storage system including a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller;
a plurality of access devices serving as a front end to the storage zones, each access device including an access program that maintains a namespace of meta data for a plurality of data items stored by clients in the distributed replicated data storage system, wherein a first access program on a first access device is configured to perform actions including:
receiving a storage request for a current data item from a client,
processing the storage request for the current data item including forwarding the storage request to the distributed replicated data storage system and receiving a stored data item identifier,
adding meta data to a current batch of data item meta data, the meta data including the stored data item identifier, the batch representing a plurality of successful storage requests for a plurality of data items,
determining whether an elapsed time since last batch of data item meta data was stored exceeds a first threshold value,
determining whether a current batch size of the current batch of data item meta data exceeds a second threshold value,
when the elapsed time since the last batch of data item meta data was stored exceeds the first threshold value or the current batch size of the current batch of data item meta data exceeds the second threshold value,
[a] storing the current batch of updated meta data as an object in the distributed replicated data storage system,
[b] receiving a batch object identifier for the stored batch of updated meta data,
[c] distributing the batch object identifier to the plurality of access devices and/or access programs.
2. The system of claim 1 wherein the distributing the batch object identifier causes the access programs on the access devices to synchronize the namespaces among the access devices.
3. The system of claim 1 wherein the storage devices are selected from the group including hard disk drives and solid state storage devices.
4. The system of claim 1 wherein the meta data includes at least two selected from the group including: data item name, data item version, data item type, data item size, data item creation date, data item storage date, data item modification date, data item owner, data item author, data item object kind, flags, parent/predecessor data item identifier, storage policy identifier.
5. The system of claim 1 wherein the distributing causes remote access devices to update their namespaces based on the updated meta data in the object associated with the batch object ID.