Method and system for optimizing data replication for large scale archives
View Patent ↗A method and system for processing query requests, including receiving, at a secondary site, a query request from a client and executing the query request to obtain an archive replica package (ARP). The method further includes making a determination that a record associated with the ARP is not stored at the secondary site and based on the determination, transmitting a request to a primary site. The method further includes, in response to the request to the primary site, receiving an archive package and a record where the archive package is associated with the record, and providing the first record to the client.
1. A method for processing query requests, comprising:
receiving, at a secondary site comprising a first query node, a query request from a first client, wherein the query request specifies a first field comprising metadata describing a first record;
executing the query request to obtain a first archive replica package (ARP) stored at the secondary site, wherein the first ARP comprises a first plurality of fields, wherein a second field of the first plurality of fields matches the first field;
making a first determination that the first record is not stored at the secondary site, wherein the first record is associated with the first ARP;
based on the first determination:
transmitting a request to a primary site comprising a second query node, wherein the primary site is located in a first geographic region and the secondary site is located in a second geographic region;
in response to the request, receiving a first archive package (AP) and the first record, wherein the first AP is associated with the first record and the first ARP; and
providing the first record to the client.
2. The method of claim 1 , further comprising:
receiving, from the primary site, a second ARP, wherein the second ARP is associated with a second AP and a second record stored on the primary site.
3. The method of claim 1 , further comprising:
receiving, at the secondary site, a second query request from a second client;
executing the second query request to obtain the first ARP;
making a second determination that the first record associated with the first ARP is stored at the secondary site;
based on the second determination:
obtaining the first record from a cache at the secondary site; and
providing the first record to the client.
4. The method of claim 1 , further comprising:
receiving, at the secondary site, a second query request from a second client;
executing the second query request to obtain a second ARP;
making a second determination that a second record associated with the second ARP is stored at the secondary site;
based on the second determination:
obtaining the second record from a cache at the secondary site; and
providing the second record to the client.
5. The method of claim 1 , wherein the secondary site comprises a plurality of ARPs organized in a hierarchy and wherein the hierarchy mirrors a hierarchy of a plurality of corresponding APs stored at the primary site, wherein the first ARP is one of the plurality of ARPs.
6. The method of claim 1 , wherein the ARP is an eXtensible Mark-up Language (XML) file.
7. The method of claim 1 , wherein the first AP comprises a second plurality of fields, wherein the first plurality of fields is a subset of the second plurality of fields.
8. The method of claim 7 , wherein the first ARP further comprises a reference to the first AP, wherein the first ARP does not include a record reference for the first record, wherein the first AP further comprises the record reference.
9. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor, enables the computer processor to:
receive, at a secondary site comprising a first query node, a query request from a first client, wherein the query request specifies a first field comprising metadata describing a first record;
execute the query request to obtain a first archive replica package (ARP) stored at the secondary site, wherein the first ARP comprises a first plurality of fields, wherein a second field of the first plurality of fields matches the first field;
make a first determination that the first record is not stored at the secondary site, wherein the first record is associated with the first ARP;
based on the first determination:
transmit a request to a primary site comprising a second query node, wherein the primary site is located in a first geographic region and the secondary site is located in a second geographic region;
in response to the request, receive a first archive package (AP) and the first record, wherein the first AP is associated with the first record and the first ARP; and
provide the first record to the client.
10. The non-transitory computer readable medium of claim 9 , further comprising computer readable program code, which when executed by the computer processor, enables the computer processor to:
receive, from the primary site, a second ARP, wherein the second ARP is associated with a second AP and a second record stored on the primary site.
11. The non-transitory computer readable medium of claim 9 , further comprising computer readable program code, which when executed by the computer processor enables the computer processor to:
receive, at the secondary site, a second query request from a second client;
execute the second query request to obtain the first ARP;
make a second determination that the first record associated with the first ARP is stored at the second site;
based on the second determination:
obtain the first record from a cache at the secondary site; and
provide the first record to the client.
12. The non-transitory computer readable medium of claim 9 , further comprising computer readable program code, which when executed by the computer processor enables the computer processor to:
receive, at the secondary site, a second query request from a second client;
execute the second query request to obtain a second ARP;
make a second determination that a second record associated with the second ARP is stored at the secondary site;
based on the second determination:
obtain the second record from a cache at the secondary site; and
provide the second record to the client.
13. The non-transitory computer readable medium of claim 9 , wherein the secondary site comprises a plurality of ARPs organized in a hierarchy and wherein the hierarchy mirrors a hierarchy of a plurality of corresponding APs stored at the primary site, wherein the first ARP is one of the plurality of ARPs.
14. The non-transitory computer readable medium of claim 9 , wherein the first AP comprises a second plurality of fields, wherein the first plurality of fields is a subset of the second plurality of fields.
15. The non-transitory computer readable medium of claim 14 , wherein the first ARP further comprises a reference to the first AP, wherein the first ARP does not include a record reference for the first record, wherein the first AP further comprises the record reference.