Upgrading on-disk format without service interruption
A logical map represents fragments from separate versions of a data object. Migration of data from a first (old) version to the second (new) version happens gradually, where write operations go to the new version of the data object. The logical map initially points to the old data object, but is updated to point to the portions of the new data object as write operations are performed on the new data object. A background migration copies data from the old data object to the new data object.
1. A method comprising:
receiving a read operation to read data from a data object;
accessing a plurality of map entries stored in a logical map, the logical map representing fragments of a first version of the data object and fragments of a second version of the data object, each map entry including a flag that is set to indicate whether a fragment is the first version of the data object or the fragment is the second version of the data object; and
reading data from each fragment represented by each of the plurality of accessed map entries including:
when said each accessed map entry includes the flag that is set to indicate a fragment is the first version of the data object, then reading the data from the first version of the data object; and
when said each accessed map entry includes the flag that is set to indicate a fragment is the second version of the data object, then reading the data from the second version of the data object.
2. The method of claim 1 , wherein the received read operation specifies a starting logical block L in the data object, the method including: accessing a first map entry from the logical map that represents a fragment containing the starting logical block L; and
reading a portion of data contained the fragment determined based on a location of the starting logical block within the fragment.
3. The method of claim 2 , wherein reading a portion of data contained the fragment is further determined based on a remaining number of blocks of the read operation.
4. The method of claim 1 , wherein a fragment of the first version of the data object comprises one or more logical blocks of the data object that are stored in the first version of the data object, wherein a fragment of the second version of the data object comprises one or more logical blocks of the data object that are stored in the second version of the data object.
5. The method of claim 1 , wherein each map entry in the logical map comprises a logical block address (LBA) of a first logical block in the fragment represented by said each map entry and a block count of the number of blocks in the fragment, wherein accessing the map entry includes searching the plurality of map entries in the logical map for a map entry having the largest logical block address that is equal to or less than the starting block L.
6. The method of claim 1 , wherein reading data from the first or second version of the data object includes accessing one or more physical data blocks on a respective data storage device of the first or second version of the data object and using physical block address (PBA) information in the accessed map entry to access the one or more physical data blocks on the respective data storage device.
7. The method of claim 1 , wherein the first version of the data object is a virtual disk having a first storage format, wherein the second version of the data object is virtual disk having a second storage format different from the first storage format.
8. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computer device, cause the computer device to:
receive a read operation to read data from a data object;
access a plurality of map entries stored in a logical map, the logical map representing fragments of a first version of the data object and fragments of a second version of the data object, each map entry including a flag that is set to indicate whether a fragment is the first version of the data object or the fragment is the second version of the data object; and
read data from each fragment represented by each of the plurality of accessed map entries including:
when said each accessed map entry includes the flag that is set to indicate a fragment is the first version of the data object, then reading the data from the first version of the data object; and
when said each accessed map entry includes the flag that is set to indicate a fragment is the second version of the data object, then reading the data from the second version of the data object.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the read operation specifies a starting logical block L in the data object, the method including:
accessing a first map entry from the logical map that represents a fragment containing the starting logical block L; and
reading a portion of data contained the fragment determined based on a location of the starting logical block within the fragment.
10. The non-transitory computer-readable storage medium of claim 9 , wherein reading a portion of data contained the fragment is further determined based on a remaining number of blocks of the read operation.
11. The non-transitory computer-readable storage medium of claim 8 , wherein a fragment of the first version of the data object comprises one or more logical blocks of the data object that are stored in the first version of the data object, wherein a fragment of the second version of the data object comprises one or more logical blocks of the data object that are stored in the second version of the data object.
12. The non-transitory computer-readable storage medium of claim 8 , wherein each map entry in the logical map comprises a logical block address (LBA) of a first logical block in the fragment represented by said each map entry and a block count of the number of blocks in the fragment, wherein accessing the map entry includes searching the plurality of map entries in the logical map for a map entry having the largest logical block address that is equal to or less than the starting block L.
13. The non-transitory computer-readable storage medium of claim 8 , wherein reading data from the first or second version of the data object includes accessing one or more physical data blocks on a respective data storage device of the first or second version of the data object and using physical block address (PBA) information in the accessed map entry to access the one or more physical data blocks on the respective data storage device.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the first version of the data object is a virtual disk having a first storage format, wherein the second version of the data object is virtual disk having a second storage format different from the first storage format.
15. An apparatus comprising:
one or more computer processors; and
a computer-readable storage medium comprising instructions for controlling the one or more computer processors to: receive a read operation to read data from a data object;
access a plurality of map entries stored in a logical map, the logical map representing fragments of a first version of the data object and fragments of a second version of the data object, each map entry including a flag that is set to indicate whether a fragment of the first version is the data object or the fragment the second version is the data object; and
read data from each fragment represented by each of the plurality of accessed map entries including:
when said each accessed map entry includes the flag that is set to indicate a fragment is the first version of the data object, then reading the data from the first version of the data object; and
when said each accessed map entry includes the flag that is set to indicate a fragment is the second version of the data object, then reading the data from the second version of the data object.
16. The apparatus of claim 15 , wherein the read operation specifies a starting logical block L in the data object, the method including:
accessing a first map entry from the logical map that represents a fragment containing the starting logical block L; and
reading a portion of data contained the fragment determined based on a location of the starting logical block within the fragment.
17. The apparatus of claim 16 , wherein reading a portion of data contained the fragment is further determined based on a remaining number of blocks of the read operation.
18. The apparatus of claim 15 , wherein each map entry in the logical map comprises a logical block address (LBA) of a first logical block in the fragment represented by said each map entry and a block count of the number of blocks in the fragment, wherein accessing the map entry includes searching the plurality of map entries in the logical map for a map entry having the largest logical block address that is equal to or less than the starting block L.
19. The apparatus of claim 15 , wherein reading data from the first or second version of the data object includes accessing one or more physical data blocks on a respective data storage device of the first or second version of the data object and using physical block address (PBA) information in the accessed map entry to access the one or more physical data blocks on the respective data storage device.
20. The apparatus of claim 15 , wherein the first version of the data object is a virtual disk having a first storage format, wherein the second version of the data object is virtual disk having a second storage format different from the first storage format.