Systems and methods for facilitating data transformation
Systems and methods are provided for facilitating the transformation of data from a tabular data set organized according to a data schema to an object based data set organized according to data ontology. The data schema and the data ontology may be used to generate a custom coding language suitable for facilitating the creation of a transform script. Error checking, type checking, autocomplete, and preview tools may be provided to permit the verification of the transform script as it is created.
1. A system for performing an integration of an origin data set into a target data set, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
receive the origin data set, the origin data set being organized according to an origin data set schema specifying a format for data of the origin data set;
receive a target data set ontology, the target data set ontology defining data objects of the target data set;
generate, according to the origin data set schema and the target data set ontology, a transform language specific to the origin data set schema and the target data set ontology;
receive transform instructions programmed in the transform language specific to the origin data set schema and the target data set ontology;
analyze the transform instructions to identify one or more errors in the transform instructions, the analysis being performed as the transform instructions are being received; and
integrate, subsequent to a correction of the one or more errors in the transform instructions, at least a portion of the origin data set into the target data set based on the corrected transform instructions.
2. The system of claim 1 , wherein to analyze the transform instructions to identify one or more errors in the transform instructions, the system is further caused to:
identify one or more first references to the origin data set schema in the transform instructions;
compare the origin data set schema to a portion of the transform instructions associated with the one or more first references to the origin data set schema;
determine, based on the comparison, whether the portion of the transform instructions associated with the one or more first references to the origin data set schema have one or more errors.
3. The system of claim 2 , wherein to analyze the transform instructions to identify one or more errors in the transform instructions, the system is further caused to:
identify one or more second references to the target data set ontology in the transform instructions;
compare the target data set ontology to a portion of the transform instructions associated with the one or more second references to the origin data set;
determine, based on the comparison, whether the portion of the transform instructions associated with the one or more second references to the target data set ontology have one or more errors.
4. The system of claim 3 , wherein the one or more second references to the target data set ontology in the transform instructions include an assignment of a data type to an object property data field.
5. The system of claim 3 , wherein the one or more second references to the target data set ontology in the transform instructions include an assignment of an object property to an object type.
6. The system of claim 1 , wherein the system is further caused to notify a user of one or more errors in the transform instructions in response to identifying the one or more errors.
7. The system of claim 6 , wherein to notify the user of the one or more errors in the transform instructions, the system is further caused to:
mark-up at least a portion of the transform instructions associated with the one or more errors in the transform instructions; and
generate a graphical user interface including the mark-up of the at least a portion of the transform instructions associated with the one or more errors in the transform instructions.
8. The system of claim 1 , wherein the system is further caused to store data transformation information about the integration, the data transformation information including information about the transform instructions and a time and date of integration.
9. The system of claim 8 , wherein to integrate the at least a portion of the origin data set into the target data set, the system is further caused to select the at least a portion of the origin data set based on the data transformation information and integrate the selected portion of the origin data set into the target data set.
10. The system of claim 1 , wherein the system is further caused to provide a transform instructions template for receiving the transform instructions.
11. The system of claim 1 , wherein the system is further caused to provide autocomplete suggestions in response to receiving the transform instructions.
12. A computer implemented method of integrating an origin data set into a target data set, the method being performed on a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, cause the computer system to perform the method, the method comprising:
receiving the origin data set, the origin data set being organized according to an origin data set schema specifying a format for data of the origin data set;
receiving a target data set ontology, the target data set ontology defining data objects of the target data set;
generating, according to the origin data set schema and the target data set ontology, a transform language specific to the origin data set schema and the target data set ontology;
receiving transform instructions programmed in the transform language specific to the origin data set schema and the target data set ontology;
analyzing the transform instructions to identify one or more errors in the transform instructions, the analysis being performed as the transform instructions are being received; and
integrating, subsequent to a correction of the one or more errors in the transform instructions, at least a portion of the origin data set into the target data set based on the corrected transform instructions.
13. The method of claim 12 , wherein the analyzing the transform instructions to identify one or more errors in the transform instructions further comprises:
identifying one or more first references to the origin data set schema in the transform instructions;
comparing the origin data set schema to a portion of the transform instructions associated with the one or more first references to the origin data set schema;
determining, based on the comparison, whether the portion of the transform instructions associated with the one or more first references to the origin data set schema have one or more errors.
14. The method of claim 13 , wherein the analyzing the transform instructions to identify one or more errors in the transform instructions further comprises:
identifying one or more second references to the target data set ontology in the transform instructions;
comparing the target data set ontology to a portion of the transform instructions associated with the one or more second references to the origin data set;
determining, based on the comparison, whether the portion of the transform instructions associated with the one or more second references to the target data set ontology have one or more errors.
15. The method of claim 14 , wherein the one or more second references to the target data set ontology in the transform instructions include an assignment of a data type to an object property data field.
16. The method of claim 14 , wherein the one or more second references to the target data set ontology in the transform instructions include an assignment of an object property to an object type.
17. The method of claim 12 , further comprising notifying a user of one or more errors in the transform instructions in response to identifying the one or more errors.
18. The method of claim 17 , wherein the notifying the user of the one or more errors in the transform instructions further comprises:
marking-up at least a portion of the transform instructions associated with the one or more errors in the transform instructions; and
generating a graphical user interface including the mark-up of the at least a portion of the transform instructions associated with the one or more errors in the transform instructions.
19. The method of claim 12 , further comprising generating autocomplete suggestions in response to receiving the transform instructions.
20. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform a method of integrating an origin data set into a target data set, the method comprising:
receiving the origin data set, the origin data set being organized according to an origin data set schema specifying a format for data of the origin data set;
receiving a target data set ontology, the target data set ontology defining data objects of the target data set;
generating, according to the origin data set schema and the target data set ontology, a transform language specific to the origin data set schema and the target data set ontology;
receiving transform instructions programmed in the transform language specific to the origin data set schema and the target data set ontology;
analyzing the transform instructions to identify one or more errors in the transform instructions, the analysis being performed as the transform instructions are being received; and
integrating, subsequent to a correction of the one or more errors in the transform instructions, at least a portion of the origin data set into the target data set based on the corrected transform instructions.