System and method for data discovery in cloud environments
A system and method for data discovery. A method includes performing a scan of a plurality of snapshots, each snapshot corresponding to a respective disk of a plurality of disks; identifying a plurality of data store files in the plurality of disks based on file metadata found during the scan; and detecting at least one data store based on the identified plurality of data store files, wherein each of the at least one data store is in a disk of the plurality of disks including one of the plurality of data store files.
1 . A method for data discovery, comprising:
analyzing data used to query disks via a plurality of interfaces in order to detect a plurality of disks in a computing environment, wherein the plurality of interfaces are used to communicate with components in the computing environment;
identifying a plurality of snapshots based on the plurality of disks, wherein identifying each snapshot includes searching a directory and identifying a most recently updated snapshot for one of the plurality of disks;
performing a scan of the plurality of snapshots, each snapshot corresponding to a respective disk of the plurality of disks;
identifying a plurality of data store files in the plurality of disks based on file metadata found during the scan; and
detecting at least one data store based on the identified plurality of data store files, wherein each of the at least one data store is in a disk of the plurality of disks including one of the plurality of data store files.
2 . The method of claim 1 , further comprising:
creating at least one engine based on a plurality of engine-relevant parameters, wherein each created engine is configured to read files having a respective database format, wherein the plurality of engine-relevant parameters is identified among data in the at least one data store; and
accessing data in the at least one data store using the created at least one engine.
3 . The method of claim 2 , further comprising
determining at least one engine type based on the plurality of engine-relevant parameters, wherein each of the at least one engine type has a configuration corresponding to a known engine used for scanning files having a respective database format, wherein the at least one engine is created based on the determined at least one engine type.
4 . The method of claim 3 , wherein each determined engine type corresponds to at least one of: a respective type of engine, and a respective version of engine.
5 . The method of claim 2 , further comprising:
mounting a copy of disk containing one of the detected at least one data store to at least one virtual machine, wherein the created at least one engine is run on the at least one virtual machine.
6 . The method of claim 5 , wherein the plurality of disks is deployed in a cloud environment, wherein the at least one virtual machine is deployed outside the cloud environment.
7 . The method of claim 1 , wherein performing the scan of the plurality of snapshots further comprises:
scanning a portion of each snapshot to determine a likelihood that the respective disk for each snapshot contains a data store, wherein a full scan is performed on each snapshot for which the determined likelihood is above a threshold.
8 . The method of claim 1 , further comprising:
implementing protective measures with respect to the at least one data store.
9 . The method of claim 8 , further comprising:
determining, for each of the at least one data store, whether the data store contains data having a predetermined data class, wherein the protective measures are implemented for each data store containing data having a predetermined data class.
10 . The method of claim 1 , further comprising:
determining, based on a backup of each disk containing one of the detected at least one data store, whether each data store is a data store that has been deleted but for which a backup exists.
11 . The method of claim 1 , further comprising:
determining, for each of the at least one data store, whether the data store is connected to any other physical or virtual machines.
12 . A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:
analyzing data used to query disks via a plurality of interfaces in order to detect a plurality of disks in a computing environment, wherein the plurality of interfaces are used to communicate with components in the computing environment;
identifying a plurality of snapshots based on the plurality of disks, wherein identifying each snapshot includes searching a directory and identifying a most recently updated snapshot for one of the plurality of disks;
performing a scan of the plurality of snapshots, each snapshot corresponding to a respective disk of the plurality of disks;
identifying a plurality of data store files in the plurality of disks based on file metadata found during the scan; and
detecting at least one data store based on the identified plurality of data store files, wherein each of the at least one data store is in a disk of the plurality of disks including one of the plurality of data store files.
13 . A system for data discovery, comprising:
a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
analyze data used to query disks via a plurality of interfaces in order to detect a plurality of disks in a computing environment, wherein the plurality of interfaces are used to communicate with components in the computing environment;
identify a plurality of snapshots based on the plurality of disks, wherein identifying each snapshot includes searching a directory and identifying a most recently updated snapshot for one of the plurality of disks;
perform a scan of the plurality of snapshots, each snapshot corresponding to a respective disk of the plurality of disks;
identify a plurality of data store files in the plurality of disks based on file metadata found during the scan; and
detect at least one data store based on the identified plurality of data store files, wherein each of the at least one data store is in a disk of the plurality of disks including one of the plurality of data store files.
14 . The system of claim 13 , wherein the system is further configured to:
create at least one engine based on a plurality of engine-relevant parameters, wherein each created engine is configured to read files having a respective database format, wherein the plurality of engine-relevant parameters is identified among data in the at least one data store; and
access data in the at least one data store using the created at least one engine.
15 . The system of claim 14 , wherein the system is further configured to
determine at least one engine type based on the plurality of engine-relevant parameters, wherein each of the at least one engine type has a configuration corresponding to a known engine used for scanning files having a respective database format, wherein the at least one engine is created based on the determined at least one engine type.
16 . The system of claim 15 , wherein each determined engine type corresponds to at least one of: a respective type of engine, and a respective version of engine.
17 . The system of claim 14 , wherein the system is further configured to:
mount a copy of disk containing one of the detected at least one data store to at least one virtual machine, wherein the created at least one engine is run on the at least one virtual machine.
18 . The system of claim 17 , wherein the plurality of disks is deployed in a cloud environment, wherein the at least one virtual machine is deployed outside the cloud environment.
19 . The system of claim 13 , wherein the system is further configured to:
scan a portion of each snapshot to determine a likelihood that the respective disk for each snapshot contains a data store, wherein a full scan is performed on each snapshot for which the determined likelihood is above a threshold.
20 . The system of claim 13 , wherein the system is further configured to:
implement protective measures with respect to the at least one data store.
21 . The system of claim 20 , wherein the system is further configured to:
determine, for each of the at least one data store, whether the data store contains data having a predetermined data class, wherein the protective measures are implemented for each data store containing data having a predetermined data class.
22 . The system of claim 13 , wherein the system is further configured to:
determine, based on a backup of each disk containing one of the detected at least one data store, whether each data store is a data store that has been deleted but for which a backup exists.
23 . The system of claim 13 , wherein the system is further configured to:
determine, for each of the at least one data store, whether the data store is connected to any other physical or virtual machines.