Engineering data analytics platforms using machine learning
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for engineering a data analytics platform using machine learning are disclosed. In one aspect, a method includes the actions of receiving data indicating characteristics of data for analysis, analysis techniques to apply to the data, and requirements of users accessing the analyzed data. The actions further include accessing provider information that indicates computing capabilities of a respective data analysis provider, analysis techniques provided by the respective data analysis provider, and real-time data analysis loads of the respective data analysis provider. The actions further include applying the characteristics of the data, the analysis techniques, the requirements of the users, and the provider information, the analysis techniques, and the real-time data analysis loads to a model. The actions further include configuring the one or more particular data analysis providers to perform the analysis techniques on the data.
1. A computer-implemented method comprising:
accessing, for each past data analysis provider of one or more past data analysis providers, historical information that includes (i) past characteristics of past data analyzed by the past data analysis provider, (ii) past computing loads during analysis of the past data by the past data analysis provider, (iii) past analysis techniques offered by the past data analysis provider, (iv) past access and security requirements implemented by the past data analysis provider for past users accessing the past data analyzed by the data analysis provider, and (v) past configurations of the respective data analysis provider used to analyze the past data;
training, using machine learning and using the historical information, a model that receives (i) given characteristics of given data, (ii) given computing loads of each of one or more given data analysis providers, (iii) given analysis techniques offered by each of the one or more given data analysis providers, (iv) given access and security requirements for given users accessing analyzed given data, and outputs data indicating (i) a subset of the one or more given data analysis providers to analyze the given data and (ii) a given configuration for the subset of the one or more given data analysis providers; and
providing the model to a recommendation generator, wherein the recommendation generator recommends based on the model both (i) one or more data analysis providers to analyze data and (ii) a configuration for the one or more data analysis providers.
2. The method of claim 1 , wherein the past characteristics of the past data include a location associated with the past data.
3. The method of claim 1 , wherein the past characteristics of the past data include a type of users associated with the past data.
4. The method of claim 1 , wherein the past analysis techniques offered by the past data analysis provider include processing capabilities associated with each past data analysis provider.
5. The method of claim 1 , wherein the model further receives historical information that indicates past usage of the past data.
6. The method of claim 1 , wherein past access and security requirements implemented by the past data analysis provider for past users accessing the past data analyzed by the data analysis provider comprise one or more of requiring users be behind a firewall, use two factor authentication, or be scanned for viruses.
7. A system comprising:
one or more computers; and
one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
accessing, for each past data analysis provider of one or more past data analysis providers, historical information that includes (i) past characteristics of past data analyzed by the past data analysis provider, (ii) past computing loads during analysis of the past data by the past data analysis provider, (iii) past analysis techniques offered by the past data analysis provider, (iv) past access and security requirements implemented by the past data analysis provider for past users accessing the past data analyzed by the data analysis provider, and (v) past configurations of the respective data analysis provider used to analyze the past data;
training, using machine learning and using the historical information, a model that receives (i) given characteristics of given data, (ii) given computing loads of each of one or more given data analysis providers, (iii) given analysis techniques offered by each of the one or more given data analysis providers, (iv) given access and security requirements for given users accessing analyzed given data, and outputs data indicating (i) a subset of the one or more given data analysis providers to analyze the given data and (ii) a given configuration for the subset of the one or more given data analysis providers; and
providing the model to a recommendation generator, wherein the recommendation generator recommends based on the model both (i) one or more data analysis providers to analyze data and (ii) a configuration for the one or more data analysis providers.
8. The system of claim 7 , wherein the past characteristics of the past data include a location associated with the past data.
9. The system of claim 7 , wherein the past characteristics of the past data include a type of users associated with the past data.
10. The system of claim 7 , wherein the past analysis techniques offered by the past data analysis provider include processing capabilities associated with each past data analysis provider.
11. The system of claim 7 , wherein the model further receives historical information that indicates past usage of the past data.
12. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
accessing, for each past data analysis provider of one or more past data analysis providers, historical information that includes (i) past characteristics of past data analyzed by the past data analysis provider, (ii) past computing loads during analysis of the past data by the past data analysis provider, (iii) past analysis techniques offered by the past data analysis provider, (iv) past access and security requirements implemented by the past data analysis provider for past users accessing the past data analyzed by the data analysis provider, and (v) past configurations of the respective data analysis provider used to analyze the past data;
training, using machine learning and using the historical information, a model that receives (i) given characteristics of given data, (ii) given computing loads of each of one or more given data analysis providers, (iii) given analysis techniques offered by each of the one or more given data analysis providers, (iv) given access and security requirements for given users accessing analyzed given data, and outputs data indicating (i) a subset of the one or more given data analysis providers to analyze the given data and (ii) a given configuration for the subset of the one or more given data analysis providers; and
providing the model to a recommendation generator, wherein the recommendation generator recommends based on the model both (i) one or more data analysis providers to analyze data and (ii) a configuration for the one or more data analysis providers.
13. The medium of claim 12 , wherein the past characteristics of the past data include a location associated with the past data.
14. The medium of claim 12 , wherein the past characteristics of the past data include a type of users associated with the past data.
15. The medium of claim 12 , wherein the past analysis techniques offered by the past data analysis provider include processing capabilities associated with each past data analysis provider.