Hybrid data analytics in on-premise and cloud computing environments
View Patent ↗Techniques described herein relate to performing hybrid data analytics, including generating and training data queries and/or taxonomies using on-premise data analytics tools, modifying the queries, and then executing the modified queries within a cloud computing environment using a query execution platform on the cloud environment. A query transformer may be used to modify the queries (e.g., NLP or text analytics queries) using operation mapping, fine-tuning, and/or search and replace listings, into modified queries compatible with the native data analytics functionalities of the cloud environment. An initial one-time data retrieval may be performed from the cloud computing environment to allow the queries to be generated and trained effectively using the on-premise tools and based on the organization's data. After transforming and validating the modified queries, these queries may be deployed within the cloud environment, and executed using a query execution platform implementing specially-designed functions built on the native cloud tools to execute the modified queries on organization's data within the cloud environment.
1 . A system, comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, by a first computer system in an on-premise computing infrastructure of an organization, a first data set comprising a first plurality of text data records;
executing a first query, using a first data analytics software tool executing on the first computer system, wherein executing the first query comprises applying a first listing of term replacements to the first data set, using a first set of replacement operations supported by the first data analytics software tool;
validating the first query, based at least in part on the execution of the first query;
transforming the first query into a second query associated with a second data analytics software tool, wherein transforming the first query comprises transforming the first listing of term replacements into a second listing of term replacements, based at least in part on the first set of replacement operations and a second set of replacement operations supported by the second data analytics software tool;
accessing a data store associated with the organization, wherein the data store is stored in a cloud computing environment separate from the on-premise computing infrastructure, the data store comprising a second plurality of text data records;
executing the second query, using the second data analytics software tool executing in the cloud computing environment, wherein executing the second query comprises applying the second listing of term replacements to the data store, using the second set of replacement operations; and
determining a set of query output records associated with the organization, based at least in part on the execution of the second query.
2 . The system of claim 1 , wherein the first data analytics software tool executing on the first computer system supports a first set of query operations, and wherein at least one of the first set of query operations is not supported by the second data analytics software tool executing in the cloud computing environment.
3 . The system of claim 1 , wherein validating the first query comprises:
rendering the first query via a user interface provided by the first data analytics software tool; and
invoking a term replacement feature of the first data analytics software tool on the first query,
wherein the second data analytics software tool executing in the cloud computing environment does not render a user interface and does not include a term replacement feature.
4 . The system of claim 3 , the operations further comprising:
validating the first listing of term replacements, using the term replacement feature of the first data analytics software tool; and
transmitting the second listing of term replacements to the cloud computing environment.
5 . The system of claim 1 , wherein validating the first query comprises receiving a validation confirmation from a user, via a user interface provided by the first data analytics software tool, and
wherein transforming the first query into the second query is triggered automatically based at least in part on received validation confirmation.
6 . The system of claim 1 , wherein receiving the first data set comprises:
retrieving, from the cloud computing environment, a first snapshot of the data store associated with the organization at a first time, and
wherein executing the second query comprises applying the second query to a second snapshot of the data store associated with the organization at a second time after the first time.
7 . The system of claim 1 , wherein transforming the first query into the second query comprises:
determining, within the first query, a first operator supported by the first data analytics software tool;
determining that the first operator is not supported by the second data analytics software tool; and
converting the first operator to a second operator supported by the second data analytics software tool.
8 . The system of claim 1 , the operations further comprising:
receiving, at the first computer system, a data taxonomy comprising a plurality of queries, including the first query, wherein each of the plurality of queries is compatible with the first data analytics software tool; and
transforming the plurality of queries into a plurality of transformed queries, including the second query, wherein each of the plurality of queries is compatible with the second data analytics software tool.
9 . The system of claim 1 , the operations further comprising:
comparing a first output of the first query, to a second output of the second query; and
determining an accuracy score associated with the transformation of the first query into the second query, based at least in part on the comparing of the first output to the second output.
10 . A method comprising:
receiving a first query compatible with a first data analytics software tool installed in a datacenter associated with an organization, the first query comprising a first listing of term replacements performed by a first set of replacement operations supported by the first data analytics software tool;
transforming the first query into a second query associated with a second data analytics software tool, wherein transforming the first query comprises transforming the first listing of term replacements into a second listing of term replacements, based at least in part on the first set of replacement operations and a second set of replacement operations supported by the second data analytics software tool;
executing the second query, in a cloud computing environment separate from the datacenter, using the second data analytics software tool, wherein executing the second query comprises applying the second listing of term replacements, using the second set of replacement operations, to a data store associated with the organization and stored within the cloud computing environment; and
determining a set of query output records associated with the organization, based at least in part on the execution of the second query.
11 . The method of claim 10 , wherein the first data analytics software tool is configured to support a first set of query operations, and wherein at least one of the first set of query operations is not supported by the second data analytics software tool executing in the cloud computing environment.
12 . The method of claim 10 , further comprising:
rendering the first query via a user interface provided by the first data analytics software tool; and
invoking a term replacement feature of the first data analytics software tool on the first query,
wherein the second data analytics software tool executing in the cloud computing environment does not render a user interface and does not include a term replacement feature.
13 . The method of claim 12 , further comprising:
validating the first listing of term replacements, using the term replacement feature of the first data analytics software tool; and
transmitting the second listing of term replacements to the cloud computing environment.
14 . The method of claim 10 , further comprising:
validating the first query, wherein the validating comprises receiving a validation confirmation from a user, via a user interface provided by the first data analytics software tool, and
wherein transforming the first query into the second query is triggered automatically based at least in part on received validation confirmation.
15 . The method of claim 10 , further comprising:
retrieving, from the cloud computing environment, a first snapshot of the data store associated with the organization at a first time, and
wherein executing the second query comprises applying the second query to a second snapshot of the data store associated with the organization at a second time after the first time.
16 . One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed by the processor, cause the processor to perform operations comprising:
receiving a first query compatible with a first data analytics software tool executing within an on-premise computing infrastructure of an organization;
receiving a first data set comprising a first plurality of text data records associated with the organization;
executing the first query, using the first data analytics software tool, wherein executing the first query comprises applying a first listing of term replacements to the first data set, using a first set of replacement operations supported by the first data analytics software tool, and outputting a first result;
transforming, based at least in part on the first result, the first query into a second query associated with a second data analytics software tool, wherein transforming the first query comprises transforming the first listing of term replacements into a second listing of term replacements, based at least in part on the first set of replacement operations and a second set of replacement operations supported by the second data analytics software tool;
executing the second query, in a cloud computing environment separate from the on-premise computing infrastructure, using the second data analytics software tool, wherein executing the second listing of term replacements, using the second set of replacement operations, comprises applying the second query to a data store associated with the organization and stored within the cloud computing environment; and
determining a set of query output records associated with the organization, based at least in part on the execution of the second query.
17 . The one or more non-transitory computer-readable media of claim 16 , wherein receiving the first data set comprises:
retrieving, from the cloud computing environment, a first snapshot of the data store associated with the organization at a first time, and
wherein executing the second query comprises applying the second query to a second snapshot of the data store associated with the organization at a second time after the first time.
18 . The one or more non-transitory computer-readable media of claim 16 , wherein transforming the first query into the second query comprises:
determining, within the first query, a first operator supported by the first data analytics software tool;
determining that the first operator is not supported by the second data analytics software tool; and
converting the first operator to a second operator supported by the second data analytics software tool.
19 . The one or more non-transitory computer-readable media of claim 16 , the operations further comprising:
receiving a data taxonomy comprising a plurality of queries, including the first query, wherein each of the plurality of queries is compatible with the first data analytics software tool; and
transforming the plurality of queries into a plurality of transformed queries, including the second query, wherein each of the plurality of queries is compatible with the second data analytics software tool.
20 . The one or more non-transitory computer-readable media of claim 16 , the operations further comprising:
comparing a first output of the first query, to a second output of the second query; and
determining an accuracy score associated with the transformation of the first query into the second query, based at least in part on the comparing of the first output to the second output.