Triggering discovery points based on change
View Patent ↗A single system merges primary data storage, data protection, and intelligence. Intelligence is provided through in-line data analytics, and data intelligence and analytics are gathered on protected data and prior analytics, and stored in discovery points, all without impacting performance of primary storage. As data is written it is automatically mirrored for data protection as part of a High Availability (HA) process. Real-time analysis is done in-line with the HA processing, enabling a variety of data analytics. Data content can be mined from within files or blocks. The gathered intelligence is used to tag objects with extended metadata, enabling both valuable search options and rapid restore options. Data recovery begins with metadata restoration, followed by near-instantaneous access to “hot” regions of data being restored, allowing site operation to continue or resume while a restore is ongoing.
1. A method for handling data, the method executed by one or more programmable data processors, the method comprising:
receiving a data access request that includes request data;
performing in-line analytics on the data access request and the request data;
storing at least one of the in-line analytics, data access request, or request data in a storage pool;
performing additional analytics on at least one of the data access request, request data, or in-line analytics to provide extended metadata; and
if a subsequent access request includes changes to the request data, storing a discovery point in the storage pool, the discovery point including a change entry and the extended metadata,
wherein the storing of the discovery point occurs at a point in time determined by a weighted percentage change factor that depends on attributes of the changes to the request data, and wherein the percentage change factor is weighted by a weighting factor that reflects an importance of the request data, thereby minimizing the storage of discovery points with unimportant data.
2. The method of claim 1 wherein the change entry in the discovery point includes at least one of an identifier for the changes to the request data, an identity of a user who initiated either the data access request or the subsequent access request, or a time of the subsequent access request.
3. The method of claim 1 wherein the discovery point further stores both the request data in the data access request and the changes in the request data.
4. The method of claim 1 wherein the weighted percentage change factor depends on an identity of a user who initiated at least one of the data access request or the subsequent access request.
5. The method of claim 1 wherein the weighted percentage change factor depends on one or more attributes of a file specified in the data access request, and wherein the attribute is at least one of a directory, file type, or file owner.
6. The method of claim 1 wherein the weighted percentage change factor depends on a relative importance of the data related to the data access request.
7. The method of claim 1 wherein the weighted percentage change factor is further dependent upon percentage change of certain identifiable subsets of the storage pool.
8. The method of claim 1 wherein the weighted percentage change factor is further dependent upon detected deviations from usage patterns indicating an increase in frequency of specific accesses.
9. The method of claim 1 wherein the weighted percentage change factor is further dependent upon a location of one or more files which contain or relate to the data access request.
10. The method of claim 1 wherein the weighted percentage change factor is further dependent upon an identity of an application that originated the data access request.
11. The method of claim 1 wherein the data access request is received from a primary node, and the extended metadata is stored in an intelligence node.
12. The method of claim 11 wherein the primary node and intelligence node are co-located.
13. The method of claim 11 wherein the primary node and intelligence node are located remotely from one another.
14. The method of claim 11 wherein a single processing node implements the primary node and intelligence node, and the intelligence node executes when the single processing node is not occupied with functions of the primary node.
15. The method of claim 1 wherein the extended metadata includes one or more of:
extended information concerning the data access request;
an identification of who initiated the data access request;
how many modifications have occurred;
an aggregate size of the modifications;
a data object owner;
an access control list; or
a time of the data access request.