Extracting data from heterogeneous sources with library conflicts
Techniques for instantiating a reference type to create a first descendant loader and a second descendant loader. The first descendant loader and the second descendant loader may be isolated loaders. The first descendant loader may load a first plugin and a first set of libraries. The first set of libraries may be needed to complete a first operation. The second descendant loader may load a second plugin and a second set of libraries. The second set of libraries may be needed to complete a second operation. The first set of libraries may conflict with the second set of libraries. A first method of the first plugin may be invoked through a plugin interface in order to complete the first operation. A second method of the second plugin may be invoked through the plugin interface to complete the second operation.
1 . A method, comprising:
instantiating a loader type to create a first descendant loader that is a first descendant of an ancestor loader;
instantiating the loader type to create a second descendant loader that is a second descendant of the ancestor loader;
loading, by the first descendant loader into an execution environment, a first plugin and a first set of one or more libraries needed to complete a first operation;
loading, by the second descendant loader into the execution environment, a second plugin and a second set of one or more libraries needed to complete a second operation;
wherein a first type comprised within the first set of one or more libraries and a second type comprised within the second set of one or more libraries (a) cannot be loaded by a same loader into the execution environment and (b) can be loaded into the execution environment using different loaders;
invoking a first method of the first plugin, the first method comprising instructions for completing the first operation; and
invoking a second method of the second plugin, the second method comprising instructions for completing the second operation,
wherein the method is executed by at least one device including a hardware processor.
2 . The method of claim 1 , wherein the first type and the second type cannot be loaded into a same execution environment based on the first type and the second type both having a same name.
3 . The method of claim 1 , wherein both the first plugin and the second plugin implement a plugin interface that is loaded by the ancestor loader into the execution environment.
4 . The method of claim 3 :
wherein the plugin interface declares an abstract method;
wherein the first method (a) implements the abstract method and (b) is invoked through the plugin interface; and
wherein the second method (a) implements the abstract method and (b) is invoked through the plugin interface.
5 . The method of claim 1 , wherein the first method comprises instructions for extracting a first data set from a first data source, and wherein the second method comprises instructions for extracting a second data set from a second data source.
6 . The method of claim 5 , further comprising:
loading, by the ancestor loader into the execution environment, a standard data model;
invoking, a third method of the first plugin, the third method comprising instructions for converting the first data set from a first data model into the standard data model; and
invoking, a fourth method of the second plugin, the fourth method comprising instructions for converting the second data set from a second data model into the standard data model.
7 . The method of claim 6 , wherein the standard data model is a generic data model.
8 . The method of claim 5 , wherein the first method further comprises instructions for converting the first data set from a first data model into a standard data model, wherein the second method further comprises instructions for converting the second data set from a second data model into the standard data model, and further comprising:
loading, by the ancestor loader into the execution environment, the standard data model.
9 . The method of claim 1 , wherein the first descendant loader is isolated from the second descendant loader, and wherein the second descendant loader is isolated from the first descendant loader.
10 . One or more non-transitory computer-readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations comprising:
instantiating a loader type to create a first descendant loader that is a first descendant of an ancestor loader;
instantiating the loader type to create a second descendant loader that is a second descendant of the ancestor loader;
loading, by the first descendant loader into an execution environment, a first plugin and a first set of one or more libraries needed to complete a first operation;
loading, by the second descendant loader into the execution environment, a second plugin and a second set of one or more libraries needed to complete a second operation;
wherein a first type comprised within the first set of one or more libraries and a second type comprised within the second set of one or more libraries (a) cannot be loaded by a same loader into the execution environment and (b) can be loaded into the execution environment using different loaders;
invoking a first method of the first plugin, the first method comprising instructions for completing the first operation; and
invoking a second method of the second plugin, the second method comprising instructions for completing the second operation.
11 . The one or more non-transitory computer-readable media of claim 10 , wherein the operations further comprise:
loading, by the ancestor loader, a plugin interface,
wherein the plugin interface is implemented by at least one of (a) the first plugin or (b) the second plugin.
12 . The one or more non-transitory computer-readable media of claim 11 :
wherein the plugin interface declares an abstract method;
wherein the first method (a) implements the abstract method and (b) is invoked through the plugin interface; and
wherein the second method (a) implements the abstract method and (b) is invoked through the plugin interface.
13 . The one or more non-transitory computer-readable media of claim 10 , wherein the first method comprises instructions for extracting a first data set from a first data source, and wherein the second method comprises instructions for extracting a second data set from a second data source.
14 . The one or more non-transitory computer-readable media of claim 13 , wherein the operations further comprise:
loading, by the ancestor loader into the execution environment, a standard data model, wherein the standard data model is a generic data model;
invoking, a third method of the first plugin, the third method comprising instructions for converting the first data set from a first data model into the standard data model; and
invoking, a fourth method of the second plugin, the fourth method comprising instructions for converting the second data set from a second data model into the standard data model.
15 . The one or more non-transitory computer-readable media of claim 10 , wherein the first descendant loader is a sibling of the second descendant loader.
16 . A system comprising:
at least one device comprising one or more hardware processors;
the system being configured to perform operations comprising:
instantiating a loader type to create a first descendant loader that is a first descendant of an ancestor loader;
instantiating the loader type to create a second descendant loader that is a second descendant of the ancestor loader;
loading, by the first descendant loader into an execution environment, a first plugin and a first set of one or more libraries needed to complete a first operation;
loading, by the second descendant loader into the execution environment, a second plugin and a second set of one or more libraries needed to complete a second operation;
wherein a first type comprised within the first set of one or more libraries and a second type comprised within the second set of one or more libraries (a) cannot be loaded by a same loader into the execution environment and (b) can be loaded into the execution environment using different loaders;
invoking a first method of the first plugin, the first method comprising instructions for completing the first operation; and
invoking a second method of the second plugin, the second method comprising instructions for completing the second operation.
17 . The system of claim 16 , wherein both the first plugin and the second plugin implement a plugin interface, wherein the first method is invoked through the plugin interface, and wherein the second method is invoked through the plugin interface.
18 . The system of claim 16 , wherein the first method comprises instructions for extracting a first data set from a first data source, and wherein the second method comprises instructions for extracting a second data set from a second data source.
19 . The system of claim 18 , wherein the operations further comprise:
loading, by the ancestor loader into the execution environment, a standard data model, wherein the standard data model is a generic data model;
invoking, through a plugin interface, a third method of the first plugin, the third method comprising instructions for converting the first data set from a first data model into the standard data model; and
invoking, through the plugin interface, a fourth method of the second plugin, the fourth method comprising instructions for converting the second data set from a second data model into the standard data model.
20 . The system of claim 16 , wherein the ancestor loader is a parent of the first descendant loader, and the second descendant loader.