Intelligently pre-placing data for local consumption by workloads in a virtual computing environment
View Patent ↗Methods and apparatus involve intelligently pre-placing data for local consumption by workloads in a virtual computing environment. Access patterns of the data by the workload are first identified. Based thereon, select data portions are migrated from a first storage location farther away the workload to a second storage location closer the workload. Migration also occurs at a time when needed by the workload during use. In this manner, bandwidth for data transmission is minimized. Latency effects created by consumption of remotely stored data is overcome as well. In various embodiments, a data vending service and proxy are situated between a home repository of the data and the workload. Together they serve to manage and migrate the data as needed. Data recognition patterns are disclosed as is apportionment of the whole of the data into convenient migration packets. De/Encryption, (de)compression, computing systems and computer program products are other embodiments.
1. In a computing system environment, a method of placing data for consumption by workloads of computing devices having hardware platforms, comprising:
identifying patterns of access by the workload to the data stored on a storage computing device at a first storage location to predetermine when and which portion of the data the workload will consume by establishing traces, the traces comprising trace data defining at least one of an order and a timing of access to particular storage blocks of the data by the workload, wherein the order reveals patterns of the storage blocks of the data being accessed in sequential, algorithmic, or back and forth manner;
based on the traces, apportioning a whole of the data into portions whereby a data access pattern of the workload is satisfied; and
migrating a first portion of the data from the storage computing device at the first storage location farther away from the workload to a second storage computing device at a second storage location closer to the workload just in time for consumption by the workload during use.
2. The method of claim 1 , further including deploying the workload first near the storage location to said identify the access patterns and second near the second storage location in a cloud computing environment, the second deploying of the workload occurring before the migrating the portion of the data.
3. The method of claim 1 , further including migrating a next portion of the data from the storage computing device at the storage location farther away from the workload to the second storage computing device at the second storage location closer to the workload at a second time later said time when just needed by the workload during use.
4. The method of claim 3 , further including destroying the migrated portion of the data at the second storage computing device at the second storage location closer to the workload after said migrating the next portion of the data.
5. The method of claim 1 , further including compressing the portion of the data before the migrating and decompressing the portion of the data after the migrating.
6. The method of claim 1 , further including encrypting the portion of the data before the migrating and decrypting the portion of the data after the migrating.
7. In a computing system environment, a method of placing data for consumption by workloads of computing devices having hardware platforms, comprising:
deploying the workload to a computing device for use;
establishing traces for patterns of access of the workload to the data stored on a storage computing device at a first storage location to predetermine when and which portion of the data the workload will consume, the traces comprising trace data defining at least one of an order and a timing of access to particular storage blocks of the data by the workload, wherein the order reveals patterns of the storage blocks of the data being accessed in sequential, algorithmic, or back and forth manner;
based on the traces, apportioning the data into portions for said consumption by the workload whereby a data access pattern of the workload is satisfied; and
migrating a first portion of the data to a second storage computing device at a second storage location closer to the workload just in time for said consumption by the workload during use.
8. The method of claim 7 , wherein the apportioning the data further includes establishing a ratio of total data size to needed data size.
9. The method of claim 8 , wherein the migrating further includes migrating the first portion of the data in an amount satisfying the ratio and later migrating a second portion of the data in a second amount also satisfying the ratio.
10. The method of claim 9 , wherein the migrating the first and second portions further includes migrating overlapping portions of a whole of the data.
11. The method of claim 7 , further including deploying the workload to a hardware platform other than a first hardware platform where the establishing traces occurred.
12. The method of claim 9 , wherein the migrating the first and second portions further includes destroying the migrated first portion of the data at the second storage computing device at the second storage location closer to the workload after said migrating the second portion of the data.
13. A computing system to place data for consumption by workloads, comprising:
at least first and second computing devices having a hardware platform with a processor and memory upon which a plurality of workloads can be configured under the scheduling control of a hypervisor;
at least first and second storage devices local to the first and second computing devices respectively for storing the data that is to be consumed by the workloads;
a data vending service configured with the first storage device to migrate portions of a whole of the data to the second storage device closer to the second computing device at a time when one of the workloads is deployed on the second computing device, but formerly deployed on the first computing device; and
a data proxy service configured with each said workload to migrate portions of a whole of the data by the data vending service whereby a data access pattern of the workload is satisfied, the data access pattern being defined by predetermined trace data defining at least one of an order and a timing of access to particular storage blocks of the data by the workload, wherein the order consists of access patterns of the workload to the storage blocks of the data in sequential, algorithmic, or back and forth manner.
14. The computing system of claim 13 , further wherein the data proxy service is in communication with the data vending service to expose storage semantics of the migrated portions of the whole of the data to appear as locally stored data to the one of the workloads when deployed on the second computing device.
15. The computing system of claim 13 , wherein the data vending service is configured as executable instructions and authenticates any of the workloads that may seek to use any of said portions of the whole of the data.
16. The computing system of claim 13 , wherein the data vending service and the data proxy service are configured to communicate via a secure computing tunnel.
17. The computing system of claim 13 , further including executable instructions to compress or encrypt any of said portions of the whole of the data before the migration by the data vending service.