Asymmetric active-active storage for hyper-converged system
A technique for providing active-active access to data over distance includes designating a first geographical site as a preferred site and designating a second geographical site as a non-preferred site. In response to a computing node at either site encountering a write request to write data to a storage object shared across both sites, the write request is handled by the preferred site, where the specified data is written to local storage at the preferred site and mirrored to local storage at the non-preferred site.
1. A method of providing asymmetric active-active access to data across multiple geographical sites, the method comprising:
operating multiple computing nodes at a first site and multiple computing nodes at a second site, the first site and the second site each having respective local storage, the local storage at the first site storing a first version of a storage object and the local storage at the second site storing a second version of the storage object;
in response to an IO splitter in a first computing node at the first site receiving a first IO request that specifies a first set of data to be written to the storage object, splitting the first IO request to direct writing of the first set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site; and
in response to an IO splitter in a second computing node at the second site receiving a second IO request specifying a second set of data to be written to the storage object, forwarding the second IO request to a computing node at the first site, the IO splitter of the computing node at the first site then splitting the second IO request to direct writing of the second set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site,
wherein the IO splitters in the first and second computing nodes are each configured to forward IO requests in a first mode and to split IO requests in a second mode.
2. The method of claim 1 , wherein the first IO request is generated by a first application instance running on the first computing node and the second IO request is generated by a second application instance running on the second computing node.
3. The method of claim 2 ,
wherein the first IO request specifies a first region of the storage object to which the first set of data is directed, and
wherein the method further comprises, prior to splitting the first IO request, requesting a write lock on the first region of the storage object from a distributed coherency engine, the distributed coherency engine extending between the first site and the second site and asserting the write lock at both the first site and the second site.
4. The method of claim 3 , wherein the distributed coherency engine has a first component running at the first site and a second component running at the second site, and wherein the method further comprises, after receiving the write lock on the first region of the storage object:
writing the first set of data to the first region in the first version of the storage object at the first site;
marking the write lock on the first region of the storage object as dirty in the second component of the distributed coherency engine; and
writing the first set of data to the first region in the second version of the storage object at the second site.
5. The method of claim 4 , further comprising:
receiving, by an IO splitter of a computing node in the second site, a read IO request to read the first region of the storage object; and
in response to the first region of the storage object being marked as dirty in the second component of the distributed coherency engine, redirecting the read IO request to the first site to obtain the first set of data from the first version of the storage object in the local storage at the first site.
6. The method of claim 5 , further comprising, in response to redirecting the read IO request to the first site:
acquiring a read lock on the first region of the storage object from the first component of the distributed coherency engine; and
after acquiring the read lock, obtaining the first set of data from the first version of the storage object in the local storage at the first site.
7. The method of claim 4 ,
wherein, a set of write IO requests directed to the first region of the storage object and arriving at the second site from the first site each have a timestamp indicating a time at which data specified in the respective IO request was written at the first site, the second site providing a first timestamp reflecting the timestamp of the write IO request whose data were most recently written to first region of the storage object at the second site, and
wherein the second component of the coherency engine is provided with a second timestamp that indicates a last time that a write lock was released from the first region of the storage object, and
wherein the method further comprises:
receiving, by an IO splitter of a computing node at the second site, a read IO request to read the first region of the storage object; and
performing, by the IO splitter, a read-splitting operation, the read-splitting operation constructed and arranged to (i) redirect the read IO request to the first site to effect reading from the first region of the storage object in the first version in response to the first timestamp indicating an earlier time than the second timestamp and (ii) directing the read IO request to effect local reading of the first region of the storage object in the second version in response to the first timestamp indicating a later time than the second timestamp.
8. The method of claim 2 , wherein the first site is a preferred site with respect to the storage object and the second site is a non-preferred site with respect to the storage object, wherein the non-preferred site is configured always to forward write IO requests directed to the storage object to the preferred site, and wherein the method further comprises:
in response to detecting that greater than a predetermined limit of write IO requests specifying data to be written to the storage object at the second site are forwarded to the first site, setting the second site to be the preferred site with respect to the storage object and setting the first site to be the non-preferred site with respect to the storage object.
9. The method of claim 2 ,
wherein the first site is a preferred site with respect to the storage object and the second site is a non-preferred site with respect to the storage object,
wherein the non-preferred site is configured always to forward write IO requests directed to the storage object to the preferred site, and
wherein the local storage at the first site and the local storage at the second site redundantly store a second storage object, the second site being a preferred site with respect to the second storage object and the first site being a non-preferred site with respect to the second storage object.
10. The method of claim 2 , wherein the storage object includes multiple storage portions, and wherein the method further comprises operating a distributed coherency engine, the distributed coherency engine including, for each of the storage portions, a respective coherency engine component for that storage portion, each coherency engine component tracking write locks and read locks for the respective storage portion.
11. The method of claim 10 , further comprising:
storing each storage portion in a respective set of storage devices at the first site; and
storing each storage portion in a respective set of storage devices at the second site.
12. The method of claim 10 , wherein the first site and the second site each include a set of appliances constructed and arranged to serialize and buffer data being exchanged between the first site and the second site, and wherein the method further comprises:
for each storage portion stored at the first site, operating a respective dedicated appliance at the first site for serializing and buffering data of the respective storage portion, each dedicated appliance at the first site implemented as a software construct running on a computing node or other computing device at the first site; and
for each storage portion stored at the second site, operating a respective dedicated appliance at the second site for serializing and buffering data of the respective storage portion, each dedicated appliance at the second site implemented as a software construct running on a computing node or other computing device at the second site.
13. The method of claim 1 ,
wherein the first site is a preferred site with respect to the storage object and the second site is a non-preferred site with respect to the storage object, and
wherein the non-preferred site is configured always to forward write IO requests directed to the storage object to the preferred site.
14. A hyper-converged system, comprising:
multiple computing nodes at a first site;
multiple computing nodes at a second site;
a network coupling the computing nodes at the first site to the computing nodes at the second site;
the first site and the second site each having respective local storage, the local storage at the first site storing a first version of a storage object and the local storage at the second site storing a second version of the storage object,
wherein the hyper-converged system is constructed and arranged to:
in response to an IO splitter in a first computing node at the first site receiving a first IO request that specifies a first set of data to be written to the storage object, split the first IO request to direct writing of the first set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site; and
in response to an IO splitter in a second computing node at the second site receiving a second IO request specifying a second set of data to be written to the storage object, forward the second IO request to a computing node at the first site, the IO splitter of the computing node at the first site then constructed and arranged to split the second IO request to direct writing of the second set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site,
wherein the IO splitters in the first and second computing nodes are each configured to forward IO requests in a first mode and to split IO requests in a second mode.
15. The hyper-converged system of claim 14 , wherein the storage object includes multiple storage portions, and wherein the system further comprises a distributed coherency engine, the distributed coherency engine including, for each of the storage portions, a respective coherency engine component for that storage portion, each coherency engine component tracking write locks and read locks for the respective storage portion.
16. The hyper-converged system of claim 15 , further comprising:
a respective set of storage devices at the first site in which each storage portion is stored; and
a respective set of storage devices at the second site in which each storage portion is stored.
17. The hyper-converged system of claim 16 , wherein the first site and the second site each include a set of appliances constructed and arranged to serialize and buffer data being exchanged between the first site and the second site, the first site including a respective dedicated appliance for each storage portion stored at the first site, the second site including a respective dedicated appliance for each storage portion stored at the second site.
18. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of multiple computing nodes, cause the control circuitry to perform a method of providing asymmetric active-active access to data across multiple geographical sites, the method comprising:
operating multiple computing nodes at a first site and multiple computing nodes at a second site, the first site and the second site each having respective local storage, the local storage at the first site storing a first version of a storage object and the local storage at the second site storing a second version of the storage object;
in response to an IO splitter in a first computing node at the first site receiving a first IO request that specifies a first set of data to be written to the storage object, splitting the first IO request to direct writing of the first set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site; and
in response to an IO splitter in a second computing node at the second site receiving a second IO request specifying a second set of data to be written to the storage object, forwarding the second IO request to a computing node at the first site, the IO splitter of the computing node at the first site then splitting the second IO request to direct writing of the second set of data both to the first version of the storage object at the first site and to the second version of the storage object at the second site,
wherein the IO splitters in the first and second computing nodes are each configured to forward IO requests in a first mode and to split IO requests in a second mode.
19. The computer program product of claim 18 , wherein the first IO request is generated by a first application instance running on the first computing node and the second IO request is generated by a second application instance running on the second computing node.
20. The computer program product of claim 19 ,
wherein the first IO request specifies a first region of the storage object to which the first set of data is directed, and
wherein the method further comprises, prior to splitting the first IO request, requesting a write lock on the first region of the storage object from a distributed coherency engine, the distributed coherency engine extending between the first site and the second site and asserting the write lock at both the first site and the second site.
21. The computer program product of claim 20 , wherein the first site is a preferred site with respect to the storage object and the second site is a non-preferred site with respect to the storage object, wherein the non-preferred site is configured always to forward write IO requests directed to the storage object to the preferred site, and wherein the method further comprises:
in response to detecting that greater than a predetermined limit of write IO requests specifying data to be written to the storage object at the second site are forwarded to the first site, setting the second site to be the preferred site with respect to the storage object and setting the first site to be the non-preferred site with respect to the storage object.