Retro-fitting synthetic full copies of data
View Patent ↗Multi-dimensional surrogation systems and methods are provided that generate at least one data surrogate using information of data and numerous data changes received from at least one data source. Embodiments described herein perform shadowing of production server databases, including creation of synthetic fulls by retrofitting log shipping to enterprise database systems, or other systems, that do not have log shipping capabilities.
1. A system comprising:
a near-line server coupled to one or more servers that include original data; and
a shadowing system coupled to the near-line server and configured to:
receive a copy of the original data;
receive delta data in a plurality of instances, wherein the delta data includes data of an incremental and differential difference between the original data; and
dynamically and continuously generate and maintain an updated version of the copy at the near-line server in near real time, wherein the generating and maintaining comprise the one or more servers writing ahead information to at least one log file in response to changes in state of the original data, the information including the delta data, the one or more servers shipping the at least one log file to the near-line server, and the near-line server receiving and applying the write-ahead logs in maintaining the updated version; and
retrofit log shipping capability for application data and other structured and unstructured data based in part on the use of an override operation of an application state to allow log files to be applied as they are received, the override operation used in part to prevent a component of the near line server from entering the application state in which no further log files can be applied.
2. The system of claim 1 , wherein the shadowing system is configured to generate and maintain asynchronously.
3. The system of claim 1 , wherein the shadowing system is configured to control the applying using modified information of the component of the near-line server.
4. The system of claim 3 , wherein the component includes one or more of structural metadata of the copy and a log file of the delta data.
5. The system of claim 4 , wherein the delta data is a plurality of log files, wherein the component is a log file of a plurality of log files.
6. The system of claim 5 , wherein the applying includes invoking an engine of the one or more servers and terminating the applying includes stalling the engine.
7. The system of claim 3 , wherein the shadowing system is configured to modify the component.
8. The system of claim 7 , wherein the component is structural metadata of the copy.
9. The system of claim 7 , wherein configured to modify comprises configured to detect a first state of the copy, wherein the first state indicates the delta data has been applied to the copy.
10. The system of claim 9 , wherein configured to modify comprises configured to change the first state to a second state, wherein the second state is a state from which another updated version can be generated by applying additional delta data to the updated version.
11. The system of claim 10 , wherein changing the first state to the second state includes modifying the structural metadata of the copy.
12. The system of claim 1 , wherein the one or more servers include a messaging and collaboration server.
13. A system comprising:
a near-line server coupled to one or more servers; and
a shadowing system coupled to the near-line server and configured to:
receive delta data that describes at least one of an incremental and differential change to original data of one or more servers;
dynamically and continuously generate and maintain an updated version of a copy of the original data at the near-line server, wherein the generating and maintaining comprise the one or more servers writing ahead information to at least one log file in response to changes in state of the original data, the information including the delta data, the one or more servers shipping the at least one log file to the near-line server, and the near-line server receiving and applying the write-ahead logs in maintaining the updated version; and
retro-fit log shipping capability for application data and other structured and unstructured data based in part on the use of an override operation of an application state to allow log files to be applied as they are received, the override operation used in part to prevent a component of the near-line server from entering the application state in which no further log files can be applied; and
control the applying using modified information of the component of the near-line server.
14. The system of claim 13 , wherein the component includes structural metadata of the copy.
15. The system of claim 13 , wherein the component includes a log file of the delta data.
16. The system of claim 13 , wherein the shadowing system is configured to modify the component.
17. The system of claim 16 , wherein the component is structural metadata of the copy.
18. The system of claim 16 , wherein configured to modify comprises configured to detect a first state of the copy, wherein the first state indicates the delta data has been applied to the copy.
19. The system of claim 18 , wherein configured to modify comprises configured to change the first state to a second state, wherein the second state is a state from which another updated version can be generated by applying additional delta data to the updated version.
20. The system of claim 19 , wherein changing the first state to the second state includes modifying the structural metadata of the copy.
21. The system of claim 19 , wherein the additional delta data is received after generating the updated version.
22. The system of claim 16 , wherein the applying includes invoking an engine of the one or more servers.
23. The system of claim 22 , wherein the shadowing system is configured to cause the engine to reference a first unapplied log file of the delta data, wherein the first unapplied log file is a first log file unapplied to the copy.
24. The system of claim 16 , wherein the delta data is a plurality of log files, wherein the component is a log file of the plurality of log files.
25. The system of claim 24 , wherein the shadowing system is configured to identify a selected log file of the plurality of log files and replicate the selected log file to form a replicated log file.
26. The system of claim 25 , wherein the selected log file is a last-received log file.
27. The system of claim 25 , wherein the shadowing system is configured to generate a modified log file by modifying information of the selected log file.
28. The system of claim 27 , wherein the applying comprises:
applying log files of the plurality of log files in sequence; and
terminating the applying in response to encountering the modified log file.
29. The system of claim 28 , wherein the terminating comprises replacing the modified log file with the replicated log file.
30. The system of claim 28 , wherein the applying includes invoking an engine of a second server and the terminating the applying includes stalling the engine.
31. The system of claim 13 , wherein the shadowing system is configured to receive the copy from the one or more servers.
32. The system of claim 31 , wherein the copy is a full copy.
33. The system of claim 31 , wherein the copy is an incremental copy.
34. The system of claim 13 , wherein the shadowing system is configured to transfer the updated version to an indexed object repository.
35. The system of claim 13 , wherein the shadowing system is configured to generate and maintain in response to at least one of an automatic trigger, a timer notification, an event notification, a poll, and a request.
36. The system of claim 35 , wherein the automatic trigger includes a trigger automatically initiated in response to at least one pre-specified parameter.
37. The system of claim 36 , wherein the automatic trigger includes content of the updated version.
38. The system of claim 35 , wherein the timer notification includes notifications corresponding to scheduled events including at least one of maintenance operations, user activities, server activities, and data population operations.
39. The system of claim 35 , wherein the event notification includes notifications corresponding to changes to data of the original data.
40. The system of claim 35 , wherein the request includes at least one of access attempts and configuration attempts to the original data by one or more of users of a second server, servers and applications.
41. The system of claim 13 , wherein the shadowing system is configured to generate and maintain the updated version with complete integrity and consistency of the original data.
42. The system of claim 13 , wherein the one or more servers include a messaging and collaboration server.
43. The system of claim 13 , wherein the original data includes one or more of application data, databases, storage groups, mailbox data, and server data.
44. The system of claim 13 , wherein the shadowing system is configured to maintain the updated version by generating another updated version by applying at least one set of log files to the updated version, the at least one set of log files received later in time than the delta data.
45. The system of claim 13 , wherein the one or more servers include one or more of local servers, remote servers, database servers, messaging servers, electronic mail servers, instant messaging servers, voice-over Internet Protocol servers, collaboration servers, Exchange Servers, portals, customer relationship management (CRM) servers, enterprise resource planning (ERP) servers, business-to-business servers, and content management servers.
46. A system comprising:
one or more near-line servers coupled to one or more servers; and
a shadowing system coupled to one or more of the near-line servers and configured to:
dynamically and continuously generate and maintain an updated version of application data at the one or more near-line servers, wherein the generating and maintaining comprise the one or more servers writing ahead information to at least one log file, the information comprising delta data that includes at least one of an incremental and differential change in state of the original data, the one or more servers shipping the at least one log file to the one or more near-line servers, and the one or more near-line servers receiving and applying the write-ahead logs in maintaining the updated version; and
retro-fit log shipping capability for application data and other structured and unstructured data based in part on use of an override operation of an application state to allow log files to be applied as they are received, the override operation to prevent a component of the one or more near-line servers from entering the application state in which no further log files can be applied;
replicate one or more of the log files associated with the delta data to form one or more replicated log files; and
generate one or more modified log files by modifying information of one or more of the log files.