Predictive orchestration of data sets within marketplace
One example method includes data orchestration. Data sets are available to users in an online marketplace. Based on requests for data sets, other data sets may be recommended. The recommended data sets are moved or copied to recommended locations. This ensures that the data sets are available to the user more quickly if subsequently requested.
1. A method, comprising:
receiving a request by an orchestration engine of an online marketplace for a data set from a user, wherein the online marketplace provide multiple data sets;
logging, by a coordination engine, metadata about the user and the request in a data set graph;
generating, by a data set recommendation engine, a recommendation for a second data set based on the data set graph;
generating, by a location recommendation engine, a recommendation for a recommended location;
providing the requested data set to the user; and
moving or copying the second data set to the recommended location based on a data set access pattern, wherein the data set access pattern identifies an order that the user typically accesses sections of data within the second data set.
2. The method of claim 1 , further comprising moving the second data set to the recommended location in anticipate of a second request from the user for the second data set.
3. The method of claim 2 , further comprising determining a current location of the second data set.
4. The method of claim 2 , wherein the recommended location and the recommendation for the second data set is based on a current user location and a current location history and current user data.
5. The method of claim 1 , further comprising moving or copying only a portion of the second data set to the recommended location.
6. The method of claim 1 , further comprising generating the recommendation for the recommended location and the recommendation for the second data set based on related users, data set metadata, permission similarities, data set size, and/or security requirements.
7. The method of claim 1 , further comprising generating the recommendation for the second data set based on similar users that requested both the requested data set and the second data set.
8. The method of claim 1 , further comprising collecting metrics that include at least one of determining how the data sets are accesses, a description of each of the data sets, who accesses the data sets, how the data sets are accessed, and metadata for each of the data sets, wherein the recommendation for the second data set is based on the metrics associated with at least the requested data set and the second data set.
9. The method of claim 1 , further comprising allowing access to a portion of the second data set, wherein at least the portion is moved to the recommended location.
10. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving a request by an orchestration engine of an online marketplace for a data set from a user, wherein the online marketplace provide multiple data sets;
logging, by a coordination engine, metadata about the user and the request in a data set graph;
generating, by a data set recommendation engine, a recommendation for a second data set based on the data set graph;
generating, by a location recommendation engine, a recommendation for a recommended location;
providing the requested data set to the user; and
moving or copying the second data set to the recommended location based on a data set access pattern, wherein the data set access pattern identifies an order that the user typically accesses sections of data within the second data set.
11. The non-transitory storage medium of claim 10 , further comprising moving the second data set to the recommended location in anticipation of a second request from the user for the second data set.
12. The non-transitory storage medium of claim 11 , further comprising determining a current location of the second data set.
13. The non-transitory storage medium of claim 11 , wherein the recommended location and the recommendation for the second data set is based on a current user location and a current location history and current user data.
14. The non-transitory storage medium of claim 10 , further comprising moving or copying only a portion of the second data set to the recommended location.
15. The non-transitory storage medium of claim 10 , further comprising generating the recommendation for the recommended location and the recommendation for the second data set based on related users, data set metadata, permission similarities, data set size, and/or security requirements.
16. The non-transitory storage medium of claim 10 , further comprising generating the recommendation for the second data set based on similar users that requested both the requested data set and the second data set.
17. The non-transitory storage medium of claim 10 , further comprising collecting metrics that include at least one of determining how the data sets are accesses, a description of each of the data sets, who accesses the data sets, how the data sets are accessed, and metadata for each of the data sets, wherein the recommendation for the second data set is based on the metrics associated with at least the requested data set and the second data set.
18. The non-transitory storage medium of claim 10 , further comprising allowing access to a portion of the second data set, wherein at least the portion is moved to the recommended location.