Method and system for real-time event journaling to provide enterprise data services
View Patent ↗A data management system or “DMS” provides a wide range of data services to data sources associated with a set of application host servers. The data management system typically comprises one or more regions, with each region having one or more clusters. A given cluster has one or more nodes that share storage. To facilitate the data service, a host driver embedded in an application server connects an application and its data to a cluster. The host driver provides a method and apparatus for capturing real-time data modifications and application state notifications and, in response, generating data transactions in the form of an event journal that is provided to the data management system. The driver functions to translate traditional file/database/block I/O into a continuous, application-aware, output data stream. Using the streams generated in this manner, the DMS offers a wide range of data services that include, by way of example only: data protection (and recovery), and disaster recovery (data distribution and data replication).
1. A method of providing a data service, comprising:
(a) providing a processor;
(b) providing a memory interoperably coupled to the processor and having computer software code stored thereon; and
(c) using the processor and the memory in combination to perform the following:
(c)(1) identifying at least one request or response expected to occur and a given action to take upon an occurrence of a given request or response;
(c)(2) for the given action identified in step (c)(1), storing at least one data event generated as a function of the given action; and
(c)(3) for the least one data event stored in step (c)(2), processing and consolidating the at least one data event with a concatenation operation;
wherein the processing and consolidating of the at least one data event results in a continuous, application-aware, data stream comprising a sequence of application consistent checkpoints;
wherein a given application consistent checkpoint is associated with information identifying the at least one data event, metadata, and at least one data change; and
wherein the processing includes determining whether a given data event of the at least one data event signifies a given application consistent checkpoint of the sequence of application consistent checkpoints and, if not, holding the given data event until the given data event is subsequently associated with the given application consistent checkpoint.
2. The method as described in claim 1 , comprising identifying an application-level event and a given action to take upon occurrence of the application-level event.
3. The method as described in claim 1 , comprising generating the given data event and storing the given data event for subsequent processing.
4. The method as described in claim 3 , comprising including the given data event, which is an internal event, into the data stream only after processing the given data event into an outbound event.
5. The method as described in claim 1 , comprising retrieving the given data event and forwarding the given data event.
6. A computer-program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, the computer-readable program code executed on one or more processors to implement a data service comprising:
(a) identifying at least one request or response expected to occur and a given action to take upon an occurrence of a given request or response;
(b) for the given action identified in step (a), storing at least one data event generated as a function of the given action; and
(c) for the least one data event stored in step (b), processing and consolidating the at least one data event with a concatenation operation;
wherein the processing and consolidating of the at least one data event results in a continuous, application-aware, data stream comprising a sequence of application consistent checkpoints;
wherein a given application consistent checkpoint is associated with information identifying the at least one data event, metadata, and at least one data change; and
wherein the processing includes determining whether a given data event of the at least one data event signifies a given application consistent checkpoint of the sequence of application consistent checkpoints and, if not, holding the given data event until the given data event is subsequently associated with the given application consistent checkpoint.
7. The computer-program product as described in claim 6 , comprising identifying an application-level event and a given action to take upon occurrence of the application-level event.
8. The computer-program product as described in claim 6 , comprising generating the given data event and storing the given data event for subsequent processing.
9. The computer-program product as described in claim 6 , comprising including the given data event, which is an internal event, into the data stream only after processing the given data event into an outbound event.
10. The computer-program product as described in claim 6 , comprising retrieving the given data event and forwarding the given data event.