IP Library Granted Patent US 7,571,168
Granted Patent B2
US 7,571,168 · App. 11/459,909 · Granted Aug 4, 2009

Asynchronous file replication and migration in a storage network

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,571,168
App. No.
11/459,909
Granted
Aug 4, 2009
Kind
B2
Abstract

File system independent techniques and mechanisms for replicating files on multiple devices are provided, migrating files from one device to another (for purposes of reliability, increased bandwidth, load balancing, capacity expansion, or reduced cost), and propagating updates from a master copy to remote replicas. The mechanisms involve work queues and asynchronous file migration daemons that operate independently from and in parallel with the primary client-server and network protocol to on-disk storage data paths.

Claims (28)

1. A computer-implemented method of performing remote file operations, the method comprising:

receiving a file task description that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the file task description;

providing a set of persistent work queues associated with a plurality of storage server computers that include one or more file task descriptions;

providing an elected work manager elected by the plurality of storage server computers and configured to accept remote file operation requests and to translate the requests into one or more corresponding tasks to be placed in the set of work queues;

providing a membership manager to determine membership of a control server computer and storage server computers that join or leave a storage system;

providing a background replication agent associated with each storage server computer in the plurality of storage server computers, the background replication agent configured to monitor a work queue in the set of work queues for the associated storage server computer, to accept requests in the work queue from the elected work manager, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, one of the requests including a migration task in which the elected work manager places a remote copy request on a work queue of a first storage server computer and the background replication agent of the first storage server computer performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage server computer and the background replication agent of the second storage server computer performs a remote delete operation and provides a confirmation to the elected work manager; and

generating a first token corresponding to a lease for a file system operation on the remote file, the first token including a first access time and volume information, in response to a first request from the first storage server computer to access the remote file, the first token allowing the first storage server computer to access the remote file based on the first access time, the first storage server computer including a timer set to a value shorter than the first access time, and wherein the first storage server computer determines whether to renew the first token when the timer times out.

2. The method of claim 1 wherein a file task description associated with replication is implemented by performing a remote copy operation with respect to the remote file, awaiting confirmation, and updating a file location map to reflect the existence of a new copy of the remote file.

3. The method of claim 1 wherein a file task description associated with the migration task is implemented by performing the remote copy operation with respect to the remote file, awaiting confirmation, updating a file location map to reflect a new file location for the remote file, and performing the remote delete operation of the original remote file.

4. The method of claim 1 wherein a file task description associated with file update propagation is implemented by performing a remote copy operation to update each existing replica of the remote file.

5. The method of claim 4 wherein a file task description associated with delete propagation is implemented by performing a remote delete operation on each existing replica.

6. A storage system comprising:

a plurality of storage nodes each comprising a storage server configured to store a plurality of files in a file system, wherein a first storage node in the plurality of storage nodes corresponds to a master storage server elected by the plurality of storage servers and including an elected work manager configured to receive a file name that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the operation, and to generate a corresponding task to be placed in a set of work queues, and wherein each storage node includes a background replication agent to communicate with the elected work manager via a work queue in the set of work queues for the associated storage node, to accept tasks in the work queue, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, wherein one of the requests is a migration task in which the elected work manager places a remote copy request on a work queue of a first storage node and the background replication agent of the first storage node performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage node and the background replication agent of the second storage node performs a remote delete operation and provides a confirmation to the elected work manager.

7. The storage system of claim 6 , wherein the operation to be performed on the remote file includes replication to be implemented by performing a remote copy operation with respect to the remote file, and wherein the elected work manager is to await confirmation and update a file location map to reflect the existence of a new copy of the remote file.

8. The storage system of claim 6 , wherein the operation to be performed on the remote file includes migration to be implemented by performing the remote copy operation, and wherein the elected work manager is to await confirmation, update a file location map to reflect a new file location for the remote file, and perform the remote delete operation of the original remote file.

9. The storage system of claim 6 , wherein the operation to be performed on the remote file includes file update propagation to be implemented by performing a remote copy operation to update each existing replica of the remote file.

10. The storage system of claim 9 , wherein the operation to be performed on the remote file includes delete propagation to be implemented by performing a remote delete operation on each existing replica.

11. An article comprising a machine-accessible storage medium including instructions that when executed cause a system to:

receive a file task description that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the file task description;

provide a set of persistent work queues associated with a plurality of storage servers that include one or more file task descriptions;

provide an elected work manager elected by the plurality of storage servers and configured to accept remote file operation requests and to translate the requests into one or more corresponding tasks to be placed in the set of work queues;

provide a background replication agent associated with each storage server in the plurality of storage servers, the background replication agent configured to monitor a work queue in the set of work queues for the associated storage server, to accept requests in the work queue from the elected work manager, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, including a migration task in which the elected work manager places a remote copy request on a work queue of a first storage server and the background replication agent of the first storage server performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage server and the background replication agent of the second storage server performs a remote delete operation and provides a confirmation to the elected work manager;

generate a first token corresponding to a lease for a file system operation on the remote file, the first token including a first access time and volume information, in response to a first request from the first storage server to access the remote file, the first token allowing the first storage server to access the file based on the first access time, the first storage server including a timer set to a value shorter than the first access time; and

determine whether to renew the first token when the timer times out.

12. The article of claim 11 , further comprising instructions that when executed enable the system to perform a remote copy operation with respect to the remote file, await confirmation, and update a file location map to reflect the existence of a new copy of the remote file.

13. The article of claim 11 , further comprising instructions that when executed enable the system to perform a remote copy operation with respect to the remote file, await confirmation, update a file location map to reflect a new file location for the remote file, and perform a remote delete of the original remote file.

14. The article of claim 11 , further comprising instructions that when executed enable the system to perform a remote copy operation to update each existing replica of the remote file.

15. The article of claim 14 , further comprising instructions that when executed enable the system to perform a remote delete operation on each existing replica.

Assignments (4)
COMPANY SPLIT Recorded Aug 20, 2024
From: HITACHI, LTD.
To: HITACHI VANTARA, LTD.
Reel/Frame 069518/0761 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2011
From: HITACHI DATA SYSTEMS CORPORATION
To: HITACHI, LTD.
Reel/Frame 025756/0339 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2010
From: PARASCALE, INC.
To: HITACHI DATA SYSTEMS CORPORATION
Reel/Frame 024892/0227 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2006
From: BAHAR, CAMERON; HOPFIELD, JOSEPH; NALAM, NAVEEN; ZAFMAN, DAVID B.; OSKOUY, RASOUL M.
To: PARASCALE, INC.
Reel/Frame 018311/0587 →