System, method, and computer program for big data integration and configuration
View Patent ↗A system, method, and computer program product are provided for big data integration and configuration. In use, a plurality of components associated with a big data solution are defined in a domain specific language utilizing one or more code generation tools. Additionally, relevant artifacts for the plurality of components associated with the big data solution are generated. Further, the relevant artifacts are packaged into a manifest, the manifest including elements required to support at least one business process. In addition, environment dependent variables of the manifest are replaced with target values associated with a target big data environment to which the manifest is to be deployed. Moreover, the manifest is deployed into operation in the target big data environment.
1. A method, comprising:
presenting to a user an editor for defining a plurality of components of a big data solution;
receiving from the user via the editor a plurality of component definitions created utilizing a domain specific language (DSL);
generating a plurality of artifacts for the plurality of received component definitions, the plurality of artifacts including:
a table with a defined schema and a partition strategy,
a structure definition that performs one or more validations,
a clean-up configuration that removes items older than a predetermined number of days, and
a schedule for implementing the clean-up configuration;
packaging the plurality of artifacts into a manifest, the manifest including:
a list of the plurality of artifacts, and
a list of applications associated with at least one business process, and
a plurality of environment dependent variables;
replacing each of the plurality of environment dependent variables of the manifest with target values associated with a target big data environment to which the manifest is to be deployed; and
deploying the manifest into operation in the target big data environment.
2. The method of claim 1 , wherein the plurality of component definitions include data structures associated with the big data solution.
3. The method of claim 1 , wherein the plurality of component definitions include entities associated with the big data solution.
4. The method of claim 1 , wherein the plurality of component definitions include events associated with the big data solution.
5. The method of claim 1 , wherein the plurality of component definitions include lookup operations associated with the big data solution.
6. The method of claim 1 , wherein defining the plurality of component definitions associated with the big data solution include defining a plurality of components across disperse technologies.
7. The method of claim 1 , wherein replacing each of the plurality of the environment dependent variables of the manifest with the target values associated with the target big data environment functions to provide an environment dependent value that is populated independent of a business process.
8. The method of claim 1 , further comprising replacing environment dependent configurations of the manifest with target configurations associated with the target big data environment to which the manifest is to be deployed.
9. The method of claim 8 , wherein replacing environment dependent configurations of the manifest with target configurations functions such that an environment dependent configuration is separated from a business configuration.
10. The method of claim 1 , wherein the domain specific language is built around various big data components for integration along with an integrated development environment to support development of the domain specific language.
11. The method of claim 1 , wherein one or more code generation tools are operable to configure and tie relevant technologies of the big data solution to create new data products in big data environments.
12. A computer program product embodied on a non-transitory computer readable medium, comprising computer code for:
presenting to a user an editor for defining a plurality of components of a big data solution;
receiving from the user via the editor a plurality of component definitions created utilizing a domain specific language (DSL);
generating a plurality of artifacts for the plurality of received component definitions, the plurality of artifacts including:
a table with a defined schema and a partition strategy,
a structure definition that performs one or more validations,
a clean-up configuration that removes items older than a predetermined number of days, and
a schedule for implementing the clean-up configuration;
packaging the plurality of artifacts into a manifest, the manifest including:
a list of the plurality of artifacts, and
a list of applications associated with at least one business process, and
a plurality of environment dependent variables;
replacing each of the plurality of environment dependent variables of the manifest with target values associated with a target big data environment to which the manifest is to be deployed; and
deploying the manifest into operation in the target big data environment.
13. The computer program product of claim 12 , wherein the computer program product is operable such that the plurality of component definitions include data structures associated with the big data solution.
14. The computer program product of claim 12 , wherein the computer program product is operable such that the plurality of component definitions include entities associated with the big data solution.
15. The computer program product of claim 12 , wherein the computer program product is operable such that the plurality of component definitions include events associated with the big data solution.
16. The computer program product of claim 12 , wherein the computer program product is operable such that the plurality of component definitions include lookup operations associated with the big data solution.
17. The computer program product of claim 12 , wherein the computer program product is operable such that defining the plurality of component definitions associated with the big data solution include defining a plurality of components across disperse technologies.
18. The computer program product of claim 12 , wherein the computer program product is operable such that replacing each of the plurality of the environment dependent variables of the manifest with the target values associated with the target big data environment functions to provide an environment dependent value that is populated independent of a business process.
19. The computer program product of claim 12 , further comprising computer code for replacing environment dependent configurations of the manifest with target configurations associated with the target big data environment to which the manifest is to be deployed.
20. A system comprising:
a memory system; and
one or more processing cores coupled to the memory system and that are each configured for:
presenting to a user an editor for defining a plurality of components of a big data solution;
receiving from the user via the editor a plurality of component definitions created utilizing a domain specific language (DSL);
generating a plurality of artifacts for the plurality of received component definitions, the plurality of artifacts including:
a table with a defined schema and a partition strategy,
a structure definition that performs one or more validations,
a clean-up configuration that removes items older than a predetermined number of days, and
a schedule for implementing the clean-up configuration;
packaging the plurality of artifacts into a manifest, the manifest including:
a list of the plurality of artifacts, and
a list of applications associated with at least one business process, and
a plurality of environment dependent variables;
replacing each of the plurality of environment dependent variables of the manifest with target values associated with a target big data environment to which the manifest is to be deployed; and
deploying the manifest into operation in the target big data environment.