Systems and methods for accessing federated data
Systems and methods are provided that allow federated data from various source systems to be accessed and analyzed through a data analysis platform. The federated data may be stored in different formats. The data analysis platform can receive the federated data in whatever format it has been stored at its respective source system. A script can be used to generate temporary representations (or temporary objects) for the federated data by transforming the federated data. Moreover, the temporary representations or temporary objects can be further transformed into a data analysis platform-specific format. A user of data analysis platform may access and/or manipulate either the temporary representations or objects as well as the data analysis platform-specific objects. Temporary objects can be transformed automatically into corresponding platform-specific objects when necessary to provide an enhanced capability or operation on the objects.
1. A system, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
executing a script for transforming federated data in one or more source computing systems to corresponding data platform objects;
generating a set of temporary objects based at least in part on the execution of the script, wherein the temporary objects each provide a first representation of some portion of the federated data;
generating a set of data platform objects based at least in part on the execution of the script, wherein the data platform objects each provide a second representation of some portion of the federated data;
providing an interface for accessing the federated data;
receiving a request, through the interface, to edit a subset of the set of temporary objects; and
modifying the subset in accordance with the request.
2. The system of claim 1 , wherein the set of temporary objects is cached in the memory.
3. The system of claim 1 , wherein the execution of the script comprises determining an ontology for an object-based data modeling framework of the system.
4. The system of claim 1 , wherein each temporary object references a portion of the federated data, and wherein each temporary object comprises a corresponding data platform object that also references the portion of the federated data.
5. The system of claim 4 , wherein the temporary object and the corresponding data platform object comprise identical properties and links.
6. The system of claim 4 , wherein the instructions further cause the system to perform:
modifying the portion of the federated data in the one or more source computing systems to trigger an update to the corresponding data platform object that references the portion of the federated data.
7. The system of claim 6 , wherein the instructions further cause the system to perform:
in response to providing the interface for accessing the federated data, receiving a search query submitted via the interface; and
generating a search result that references the portion of the federated data, wherein the update is triggered in response to the search result.
8. The system of claim 1 , wherein the instructions further cause the system to perform:
in response to providing the interface for accessing the federated data, receiving a request to apply a promotion to a temporary object through the interface; and
automatically applying a corresponding promotion to a corresponding data platform object.
9. The system of claim 8 , wherein the instructions further cause the system to perform:
in response to providing the interface for accessing the federated data, receiving a request to access the temporary object via the interface; and
obtaining and providing the corresponding data platform object through the interface.
10. The system of claim 8 , wherein the instructions further cause the system to perform:
storing metadata describing the promotion applied to the temporary object.
11. A computer-implemented method, comprising:
receiving federated data from one or more source computing systems into a data analysis platform;
executing a script for transforming the federated data to corresponding data platform objects;
generating temporary objects based at least in part on the execution of the script, wherein the temporary objects each provide a first representation of some portion of the federated data;
generating data platform objects based at least in part on the execution of the script, wherein the data platform objects each provide a second representation of some portion of the federated data;
providing an interface for accessing the federated data via the data analysis platform; receiving a request, through the interface, to edit a subset of temporary objects; and
modifying the subset in accordance with the request.
12. The computer-implemented method of claim 11 , further comprising ingesting the data platform objects into a database of the data analysis platform.
13. The computer-implemented method of claim 12 , further comprising re-ingesting the data platform objects into the database of the data analysis platform upon a change being performed on the portion of the federated data to which the data platform objects correspond.
14. The computer-implemented method of claim 11 , wherein each of the temporary objects corresponds to each of the data platform objects, and wherein each corresponding temporary object and data platform object reference the same portion of the federated data.
15. The computer-implemented method of claim 14 , wherein the temporary objects and the corresponding data platform objects have identical properties and links.
16. The computer-implemented method of claim 11 , further comprising triggering an update to the data platform objects corresponding to the portion of the federated data upon a change to the portion of the federated data at the one or more source computing systems.
17. The computer-implemented method of claim 11 , wherein generating the data platform objects and generating the temporary objects comprises executing a script adapted to determine a schema of the federated data.
18. The computer implemented method of claim 17 , wherein generating the data platform objects and generating the temporary objects comprises executing the script adapted to determine an ontology for an object-based data modeling framework of the data analysis platform.
19. The computer-implemented method of claim 11 , further comprising automatically applying promotions to the data platform objects corresponding to the temporary objects upon promotions being applied to the temporary objects.
20. The computer-implemented method of claim 19 , further comprising storing metadata describing the promotions applied to the temporary object.