Method and system for migrating data
View Patent ↗Data is migrated from an original host storage system to another replacement host storage system with minimal client impact on a network. An original host storage system is connected directly to the replacement host storage system which is in turn connected to the network. Data migration occurs while allowing a client to access data in the original host storage system. When access is requested, the replacement host storage system determines if it resides locally or in the original host storage system. Depending on where the data resides, either the replacement host storage system acts on it or obtains it from the original host storage system.
1. A method of migrating data from at least one original host storage system to a replacement host storage system on a network, comprising:
connecting a replacement host storage system to a network, and connecting an original host storage system to the replacement host storage system;
migrating data from the original host storage system to the replacement host storage system;
receiving at the replacement host storage system at least one request from a client on the network concerning data stored in at least one of the replacement host storage system and the original host storage system;
determining if the data requested has been migrated from the original host storage system to the replacement host storage system; and
if the data requested has been migrated, acting on the client request, and if the data has not been migrated, searching for the data on the original host storage system, copying the data to the replacement host storage system and acting on the client request.
determining whether additional files or directory data to be migrated exist on the original host storage system; and
if there are no additional files or directory data existing which require migration, terminating data migration.
2. The method of claim 1 , wherein the original host system is connected to the network, and further comprising disconnecting the host system from the network.
3. The method of claim 1 , wherein the original host system is connected to the network, and further comprising changing the identity of the original host system.
4. The method of claim 1 , further comprising:
building a database at the replacement host storage system indicative of what data has been migrated from the original host storage system; and if the data has been migrated, having the replacement host storage system file system act on the request.
5. The method of claim 1 , further comprising:
building a database in the replacement host storage system indicative of what data has been migrated from the original host storage system; and if the data has not been migrated, determining original host storage system information about the file and sending the request to the original host storage system for further processing.
6. The method of claim 5 , further comprising:
determining if the data subject to the request is a file or a directory;
if the data is a file, storing the data in the replacement host storage system file system, and updating data migration information in the database of the replacement host storage system to indicate more data has been migrated and stored;
if the data is a directory, creating files and subdirectories in the replacement host storage system based on information returned from the original host storage system, creating new records in the database in the replacement host storage system for all newly created files and directories based on information from the original host storage system, and updating information in the replacement host storage system database to indicate that the data has been fully copied; and
having the replacement host storage system file system act on the request.
7. The method of claim 1 , wherein said at least one original host storage system comprises multiple host storage systems.
8. The method of claim 1 , further comprising:
(a) determining whether additional files or directories data to be migrated exist on the original host storage system;
(b) if additional files or directories data exist, determining which file or directory data is next to be retrieved;
(c) retrieving the information about the file or directory from the database;
(d) determining whether the file or directory data is already stored locally; and
(e) if the file or directory data is already stored on the replacement host storage system, returning to step (a).
9. The method of claim 8 , further comprising:
if at step (d) it is determined the file or directory data is not already stored in the replacement host storage system;
determining if the data subject to the request is a file or a directory;
if the data is a file, storing the data in the replacement host storage system file system, and updating data migration information in the database of the replacement host storage system to indicate more data has been migrated and stored;
if the data is a directory, creating files and subdirectories in the replacement host storage system based on information returned from the original host storage system, creating new records in the database in the replacement host storage system for all newly created files and directories based on information from the original host storage system, and updating information in the replacement host storage system database to indicate that the data has been fully copied; and
forwarding the request to the replacement host storage system file system.
10. The method of claim 1 , wherein said request from a client is at least one of an NFS request, a CIFS request, an FTP request, and another protocol request made of the original host storage system.
11. The method of claim 1 , further comprising:
retrieving the Access Control Lists for directories and files on the original host storage system file system to be migrated to the replacement host storage system; and
for directories and files that are moved or renamed on the replacement host storage system, storing the original location on the original host storage system during data migration.
12. The method of claim 1 further comprising:
retrieving the attributes of a remote file having data to be migrated from the original host storage system to the replacement host storage system, and storing the attributes in a database in the replacement host storage system;
determining if the data in the file is linked to only one file or more than one file; and
if the data is linked to only one file, creating the file in the replacement host storage system and migrating the data.
13. A method of migrating data from at least one original host storage system to a replacement host storage system on a network, comprising:
connecting a replacement host storage system to a network, and connecting an original host storage system to the replacement host storage system;
migrating data from the original host storage system to the replacement host storage system;
receiving at the replacement host storage system at least one request from a client on the network concerning data stored in at least one of the replacement host storage system and the original host storage system;
determining if the data requested has been migrated from the original host storage system to the replacement host storage system;
if the data requested has been migrated, acting on the client request; and
if the data has not been migrated, searching for the data on the original host storage system, copying the data to the replacement host storage system and acting on the client request, said copying comprising;
retrieving attributes of a remote file having data to be migrated from the original host storage system to the replacement host storage system, and storing the attributes in a database in the replacement host storage system;
determining if the data in the file is linked to only one file or more than one file, and if the data is linked to only one file, creating the file in the replacement host storage system and migrating the data;
if the data in the file is linked to more than one file, looking up the file id and file system id from the retrieved attributes in the database in the replacement host system;
determining if the file id and file system id are found in the database;
if the file id and file system id are found in the database, retrieving the replacement host system identifier for the file and creating a hard link to the file associated with the file system identifier retrieved; and
if the file id and file system id are not found in the database, creating the file in the replacement host storage system, retrieving the identifier for the file from the replacement host storage system file system, and storing the retrieved identifier for the file in the database keyed by the file id and file system id retrieved from the attributes.
14. The method of claim 13 , further comprising:
building a database in the replacement host storage system indicative of what data has been migrated from the original host storage system; and if the data has not been migrated, determining original host storage system information about the file and sending the request to the original host storage system for farther processing.
15. The method of claim 13 , further comprising:
retrieving an Access Control List for directories and files on the original host storage system file system to be migrated to the replacement host storage system; and
for directories and files that are moved or renamed on the replacement host storage system, storing the original location on the original host storage system during data migration.
16. The method of claim 13 , further comprising:
(a) determining whether additional files or directories data to be migrated exist on the original host storage system;
(b) if additional files or directories data exist, determining which file or directory data is next to be retrieved;
(c) retrieving the information about the file or directory from the database;
(d) determining whether the file or directory data is already stored locally; and
(e) if the file or directory data is already stored on the replacement host storage system, returning to step (a); and
(f) if there are no additional files or directory data existing which require migration, terminating data migration.
17. A method of migrating data from at least one original host storage system to a replacement host storage system on a network, comprising:
connecting a replacement host storage system to a network, and connecting an original host storage system to the replacement host storage system;
migrating data from the original host storage system to the replacement host storage system;
receiving at the replacement host storage system at least one request from a client on the network concerning data stored in at least one of the replacement host storage system and the original host storage system;
in the event of multiple client requests, processing client requests in accordance with a queue and processing threads;
for a thread processing a request, instructing the thread to notify the next thread if it pends;
if the thread processing a request pends, terminating processing by that thread and beginning processing the next thread in the queue, and returning the thread that pends to the end of the queue to later process the request:
for each request, determining if the data requested has been migrated from the original host storage system to the replacement host storage system; and
if the data requested has been migrated, acting on the client request, and if the data has not been migrated, searching for the data on the original host storage system, copying the data to the replacement host storage system and acting on the client request.
18. The method of claim 17 , further comprising:
building a database in the replacement host storage system indicative of what data has been migrated from the original host storage system; and if the data has not been migrated, determining original host storage system information about the file and sending the request to the original host storage system for further processing.
19. The method of claim 17 , further comprising:
retrieving an Access Control Lists for directories and files on the original host storage system file system to be migrated to the replacement host storage system; and
for directories and files that are moved or renamed on the replacement host storage system, storing the original location on the original host storage system during data migration.
20. The method of claim 17 , further comprising:
(a) determining whether additional files or directories data to be migrated exist on the original host storage system;
(b) if additional files or directories data exist, determining which file or directory data is next to be retrieved;
(c) retrieving the information about the file or directory from the database;
(d) determining whether the file or directory data is already stored locally; and
(e) if the file or directory data is already stored on the replacement host storage system, returning to step (a); and
(f) if there are no additional files or directory data existing which require migration, terminating data migration.