IP Library Granted Patent US 10,877,682
Granted Patent B2
US 10,877,682 · App. 16/244,930 · Granted Dec 29, 2020

Non-disruptive cross-protocol live data migration

Inventors: Senthil Kumar Veluswamy (Bangalore, IN); Ayyanar Perumal (Bangalore, IN)
Assignee: Western Digital Technologies, Inc.
G06F3/0647G06F3/061G06F3/067G06F3/0679G06F12/0253
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,877,682
App. No.
16/244,930
Granted
Dec 29, 2020
Kind
B2
Abstract

A non-disruptive cross-protocol live data migration process, such as from a legacy SAN storage network to a modern NVMe-oF storage network, includes maintaining a bitmap to track the transferred blocks at migration time, and utilizing the bitmap for intelligent I/O routing decisions that take into account characteristics of SSD-type NVM storage, such as endurance characteristics associated with solid state type memory. Furthermore, protocol command translations and multipath layer capabilities such as I/O queuing are advantaged at migration time to hide the underlying complexities related to a cross-protocol mass data migration, to enable a non-disruptive data migration process intended to avoid application downtime.

Claims (82)

1. A method for data migration from an existing Storage Area Network (SAN) storage network to a Non-Volatile Memory Express over fabric (NVMe-oF) storage network, the method comprising:

establishing a storage initiator session from a Non-Volatile Memory Express (NVMe) storage array to a SAN storage array;

discovering, from the initiator session, a logical unit (LU) on the SAN storage array;

creating a name-space, on the NVMe storage array, corresponding to the discovered LU;

exposing the name-space to a host;

configuring a multipath layer from the host to the NVMe storage array;

disconnecting the host access to the SAN storage array, whereby application inputs/outputs (I/Os) are queued at the multipath layer;

restarting the queued I/Os, now to the NVMe storage array; and

initiating, via the initiator session, data transfer from the SAN storage array LU to the NVMe storage array name-space.

2. The method of claim 1 , further comprising:

maintaining a bitmap in the NVMe storage array to track transferred data blocks; and

routing read I/Os and write I/Os based on a status of corresponding Logical Block Addresses (LBAs).

3. The method of claim 2 , wherein routing write I/Os comprises:

receiving from the host a write request corresponding to a particular LBA;

determining from the bitmap whether the particular LBA is already transferred to the NVMe storage array;

responsive to the particular LBA being already transferred to the NVMe storage array, writing the particular LBA to the NVMe storage array; and

responsive to the particular LBA not being already transferred to the NVMe storage array:

updating the bitmap so that the particular LBA will not be transferred from the SAN storage array to the NVMe storage array; and

writing the particular LBA to the NVMe storage array.

4. The method of claim 2 , wherein routing read I/Os comprises:

receiving from the host a read request corresponding to a particular LBA;

determining from the bitmap whether the particular LBA is present in the NVMe storage array;

responsive to the particular LBA being present in the NVMe storage array, serving the particular LBA to the host from the NVMe storage array; and

responsive to the particular LBA not being present in the NVMe storage array:

fetching the particular LBA from the SAN storage array via the initiator session; and

serving the particular LBA to the host.

5. The method of claim 1 , wherein restarting the queued I/Os to the NVMe storage array is performed prior to an end of a Small Computer System Interface (SCSI) timeout period.

6. The method of claim 1 , further comprising:

after disconnecting the host access to the SAN storage array, handling host I/Os at the NVMe storage array.

7. The method of claim 1 , further comprising:

translating a Small Computer System Interface (SCSI)-based command to a NVMe-based command for submission to the NVMe storage array.

8. The method of claim 1 , further comprising:

translating a Small Computer System Interface (SCSI)-based error handling exception to a NVMe-based error handling exception; and

during data transfer, notifying the host of the NVMe-based error handling exception.

9. The method of claim 1 , further comprising:

during data transfer, the NVMe storage array receiving from the host an error handling exception; and

the NVMe storage array notifying the SAN storage array of the error handling exception via a Small Computer System Interface (SCSI)-based command.

10. The method of claim 1 , further comprising:

during data transfer, and in response to an external command, limiting inputs/outputs per second (IOPS) from the host.

11. The method of claim 1 , wherein creating the name-space comprises:

creating the name-space based on at least one parameter from a group of parameters, corresponding to flash-blocks of the NVMe storage array, consisting of program-erase count, disk-writes-per-day, and READ-DISTURB count.

12. The method of claim 11 , wherein creating the name-space further comprises:

determining whether the name-space spans multiple solid-state memory devices, of the NVMe storage array, having one or more different corresponding parameters from the group of parameters; and

creating the name-space from the multiple solid-state memory devices based on the one or more different corresponding parameters.

13. The method of claim 11 , further comprising:

overprovisioning the quantity of flash-blocks available for garbage collection.

14. A method for live data migration between a Storage Area Network (SAN)-based storage network and a Non-Volatile Memory Express (NVMe)-based storage network, the method comprising:

maintaining a bitmap in an NVMe storage array to track data blocks transferred from a SAN storage array, including:

tracking in a field in the bitmap, during the live data migration, Drive Writes Per Day (DWPD) associated with the NVMe storage array, and

determining, based on the bitmap field and during the live data migration, how many migration-based write inputs/outputs (I/Os) and how many host-based write I/Os to process; and

routing read I/Os and write I/Os based on a status of corresponding Logical Block Addresses (LBAs).

15. The method of claim 14 , wherein routing read I/Os comprises:

receiving a read request corresponding to a particular LBA;

determining from the bitmap whether the particular LBA is present in the NVMe storage array;

responsive to the particular LBA being present in the NVMe storage array, servicing the read request from the NVMe storage array; and

responsive to the particular LBA not being present in the NVMe storage array:

fetching the particular LBA from the SAN storage array via an Internet Small Computer System Interface (iSCSI) initiator session; and

serving the particular LBA in response to the read request.

16. The method of claim 14 , further comprising:

based on the bitmap, controlling garbage collection to avoid re-writes and repeated-writes on a same flash-block.

17. The method of claim 14 , further comprising:

creating in a journaling function a persistence setting corresponding to the bitmap; and

in response to a data transfer failure, restoring the bitmap using the journaling function.

18. A system of storage array networks comprising:

means for establishing a storage initiator session from a Non-Volatile Memory Express (NVMe) storage array to a SAN storage array;

means for discovering, from the initiator session, a logical unit (LU) on the SAN storage array;

means for creating a name-space, on the NVMe storage array, corresponding to the discovered LU;

means for exposing the name-space to a host;

means for configuring a multipath layer from the host to the NVMe storage array;

means for disconnecting the host access to the SAN storage array, whereby application inputs/outputs (I/Os) are queued at the multipath layer;

means for restarting the queued I/Os, now to the NVMe storage array; and

means for initiating, via the initiator session, data transfer from the SAN storage array LU to the NVMe storage array name-space.

19. The method of claim 2 , wherein maintaining a bitmap comprises:

tracking in a field in the bitmap, during the data migration, Drive Writes Per Day (DWPD) associated with the NVMe storage array; and

determining, based on the bitmap field and during the data migration, how many migration-based write I/Os and how many host-based write I/Os to process.

20. The method of claim 14 , wherein routing write I/Os comprises:

receiving a write request corresponding to a particular LBA;

determining from the bitmap whether the particular LBA is already transferred to the NVMe storage array;

responsive to the particular LBA being already transferred to the NVMe storage array, writing the particular LBA to the NVMe storage array; and

responsive to the particular LBA not being already transferred to the NVMe storage array:

updating the bitmap so that the particular LBA will not be transferred from the SAN storage array to the NVMe storage array; and

writing the particular LBA to the NVMe storage array.

Assignments (5)
PATENT COLLATERAL AGREEMENT - A&R LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 064715/0001 →
PATENT COLLATERAL AGREEMENT - DDTL LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 067045/0156 →
RELEASE OF SECURITY INTEREST AT REEL 052915 FRAME 0566 Recorded Feb 8, 2022
From: JPMORGAN CHASE BANK, N.A.
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 059127/0001 →
SECURITY INTEREST Recorded Feb 6, 2020
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A., AS AGENT
Reel/Frame 052915/0566 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2019
From: VELUSWAMY, SENTHIL KUMAR; PERUMAL, AYYANAR
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 047959/0358 →
Continuity (1)
Related Publication 20200225863A1 · Jul 16, 2020
Cited By (1)
US 12,481,872