System for moving real-time data events across a plurality of devices in a network for simultaneous data protection, replication, and access services
View Patent ↗A data management system or “DMS” provides a wide range of data services to data sources associated with a set of application host servers. The data management system typically comprises one or more regions, with each region having one or more clusters. A given cluster has one or more nodes that share storage. To facilitate the data service, a host driver embedded in an application server connects an application and its data to a cluster. The host driver provides a method and apparatus for capturing real-time data transactions in the form of an event journal that is provided to the data management system. The driver functions to translate traditional file/database/block I/O into a continuous, application-aware, output data stream. Using the streams generated in this manner, the DMS offers a wide range of data services that include, by way of example only: data protection (and recovery), disaster recovery (data distribution and data replication), data copy, and data query and access.
1. A data management system, comprising:
a server node coupled to a data storage;
a data manager executing on the server node;
a data source comprising an embedded application-aware host driver that captures real-time data transactions;
a continuous application-aware output data stream that originates from the embedded application-aware host driver and is received by the data manager;
wherein the received continuous application-aware output data stream comprises, for an event associated with the data source:
event-identifying data;
metadata; and
data changes;
wherein the data manager preserves a data history of the data source by translating the received continuous application-aware output data stream into an object-oriented data structure, wherein the object-oriented data structure includes unformatted records, the unformatted records include a sequence of binary record chunks, the record chunks append to a binary data page with a header that specifies the length of the unformatted record, and wherein file object associated records include sequences of meta-information which include updates to an associated file object, the file object associated records track a location and a length of each modification to the associated file object, thereby forming a translated data stream, and storing the translated data stream in an object store on the data storage; and
wherein the data manager provides one or more data services via the data history.
2. The data management system of claim 1 wherein the one or more data services include a data service that involves a pair of endpoints between which the application-aware real time event data stream is exchanged.
3. The data management system of claim 2 wherein the pair of endpoints comprise the data source as a first endpoint and a given node in the set of server nodes as a second endpoint.
4. The data management system of claim 2 wherein the pair of endpoints comprise a node in the set of one or more server nodes as a first endpoint, and a node in another set of nodes as a second endpoint, and wherein the data service distributes the application-aware real-time event data stream between the pair of endpoints.
5. The data management system of claim 2 wherein the pair of endpoints comprise a node in the set of one or more server nodes as a first endpoint, and an external data source as the second endpoint.
6. The data management system of claim 1 wherein the one or more data services include one of: data protection, data distribution, data replication, data copy, policy-based archive management, policy-based hierarchical management, and policy- based life-cycle management.
7. The data management system of claim 1 wherein the data service is one of: an auditing function and a compliance function.
8. The data management system of claim 1 wherein the data source is a data set associated with an application.
9. The data management system of claim 1 wherein a plurality of server nodes share the data storage.
10. The data management system of claim 9 wherein the plurality of server nodes perform parallel access to the data storage.
11. The data management system of claim 9 wherein each server node of the plurality of server nodes monitors the operative health of the other server nodes of the plurality of server nodes.