Systems and methods for importing data from electronic data files
Computer implemented systems and methods are disclosed for importing data from electronic data files. In accordance with some embodiments, source electronic data files are received at a data importation system and managed by the data importation system. The data importation system may load detector/transformer plugins and determined whether any of the load detector/transformer plugins apply to the received source electronic data files. The data importation system may also generate transformed data using a transformer corresponding to a detector that applies to the source electronic data files. The data importation system may import the transformed data into at least one of a data analysis system and a database.
1. A system comprising:
one or more memory devices that store a set of instructions; and
at least one processor that executes the set of instructions to perform operations for generating an updated transformed electronic data file using a detector/transformer plugin, the detector/transformer plugin including a detector and a transformer, the detector configured to detect whether an updated electronic source data file is compatible with the transformer, the transformer configured to transform the updated source electronic data file into the updated transformed electronic data file, the operations comprising:
receiving the source electronic data file;
receiving an update to the source electronic data file as an updated electronic source data file;
loading the detector/transformer plugin from a database based on a detection by the detector of the detector/transformer plugin that the updated electronic source data file is compatible with the transformer, the detection based on a file type of the updated electronic source data file;
performing the transforming of the updated source electronic data file into the updated transformed electronic data file using the transformer of the detector/transformer plugin; and
providing the updated transformed electronic data file for importing into a data analysis system.
2. The system of claim 1 , wherein an order in which detectors of a plurality of detector/transformer plugins in the database are determined to apply to the updated source electronic data file is determined based on priorities assigned to the plurality of detector/transformer plugins.
3. The system of claim 1 , the operations further comprising storing transformed data in the updated transformed electronic data file.
4. The system of claim 1 , the operations further comprising:
receiving a request to delete the source electronic data file; and
deleting the source electronic data file.
5. The system of claim 4 , the operations further comprising providing a request to the data analysis system to delete transformed data corresponding to the source electronic data file.
6. A method comprising:
performing operations for generating an updated transformed electronic data file using a detector/transformer plugin, the detector/transformer plugin including a detector and a transformer, the detector configured to detect whether an updated electronic source data file is compatible with the transformer, the transformer configured to transform the updated source electronic data file into the updated transformed electronic data file, the operations comprising:
receiving the source electronic data file;
receiving an update to the source electronic data file as an updated electronic source data file;
loading the detector/transformer plugin from a database based on a detection by the detector of the detector/transformer plugin that the updated electronic source data file is compatible with the transformer, the detection based on a file type of the updated electronic source data file; and
performing the transforming of the updated source electronic data file into the updated transformed electronic data file using the transformer of the detector/transformer plugin; and
providing the updated transformed electronic data file for importing into data analysis system, one or more modules incorporated into one or more memories of a computer system to configure one or more processors of the computer system to perform the generating of the updated transformed electronic data file using the detector/transformer plugin.
7. The method of claim 6 , wherein an order in which detectors of a plurality of detector/transformer plugins in the database are determined to apply to the updated source electronic data file is determined based on priorities assigned to the plurality of detector/transformer plugins.
8. The method of claim 6 , the operations further comprising storing transformed data in the updated transformed electronic data file.
9. The method of claim 6 , the operations further comprising:
receiving a request to delete the source electronic data file; and
deleting the source electronic data file.
10. The method of claim 9 , the operations further comprising providing a request to the data analysis system to delete transformed data corresponding to the source electronic data file.
11. The method of claim 6 , further comprising providing the transformed updated electronic data file to the data analysis system.
12. A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor of a machine, cause the machine to perform operations for generating an updated transformed electronic data file using a detector/transformer plugin, the detector/transformer plugin including a detector and a transformer, the detector configured to detect whether an updated electronic source data file is compatible with the transformer, the transformer configured to transform the updated source electronic data file into the updated transformed electronic data file, the operations comprising:
receiving a source electronic data file;
receiving an update to the source electronic data file as an updated electronic source data file;
loading the detector/transformer plugin from a database based on a detection by the detector of the detector/transformer plugin that the updated electronic source data file is compatible with the transformer, the detection based on a file type of the updated electronic source data file;
performing the transforming of the updated source electronic data file into the updated transformed electronic data file using the transformer of the detector/transformer plugin; and
providing the updated transformed electronic data file for importing into the data analysis system.