Data management in a distributed computing environment
A method for managing data in a distributed computing environment comprises receiving, on a message bus, a request to communicate data between a first system hosting data and a second system accessing data, referencing a policy defining rules for sharing data between the first system and the second system, compiling, based on the policy, a message indicating subscribers to receive the data to publish, and publishing, on the message bus, the compiled message.
1. A method for managing data in a distributed computing environment, the method comprising:
receiving, by a computing device comprising a processor, a request to communicate data between a first system hosting data and a second system accessing data;
sending, by the computing device, a query to a policy interface to access rules that establish policies for sharing data between the first system and the second system, the policies comprising a first policy governing transformation of the data from the first system, and a second policy governing a communication protocol to be used; and
transforming, according to the first policy, the data from the first system, and communicating the transformed data to the second system according to the communication protocol of the second policy.
2. The method of claim 1 , wherein the request is received from the first system to initiate propagation of the data from the first system to the second system.
3. The method of claim 1 , wherein the request is received from the second system to access data stored on the first system.
4. The method of claim 1 , further comprising
iterating a base table and an archive table containing information about the first system hosting data and the second system accessing data;
populating a results table by performing table difference operations between the base table and the archive table; and
extracting, out of the results table, identification of data to publish on a message bus.
5. The method of claim 4 , further comprising archiving the results table for future reference.
6. The method of claim 1 , wherein the policies further comprise a policy that controls a security protocol for transmitting data between the first system and the second system.
7. The method of claim 1 , further comprising transmitting a further policy to the second system,
wherein the further policy defines a duration that the transformed data received by the second system may be stored on the second system.
8. The method of claim 1 , wherein the policies comprise a further policy specifying that systems matching a characteristic are allowed to access data hosted by the first system.
9. The method of claim 1 , further comprising:
compiling, based on a policy of the policies, a message indicating subscribers to receive the data to publish; and
publishing, on a message bus, the compiled message.
10. A non-transitory storage medium comprising instructions that upon execution cause a system to:
receive from a first system hosting data, a message that indicates:
an identification of the first system; and
a description of the hosted data, residing on the first system, being made available for publication on a data bus;
receive a request to communicate data between the first system and a second system;
send a query to a policy interface to access rules that establish policies for sharing data between the first system and the second system, the policies comprising a first policy governing transformation of the data from the first system, and a second policy governing a communication protocol to be used; and
transform, according to the first policy, the data from the first system, and communicate the transformed data to the second system according to the communication protocol of the second policy.
11. The system of claim 10 , wherein the policies further comprise an encryption policy to encrypt data communicated between the first system and the second system.
12. The non-transitory storage medium of claim 10 , wherein the policies further comprise a policy that controls a security protocol for transmitting data between the first system and the second system.
13. The non-transitory storage medium of claim 10 , wherein the instructions upon execution cause the system to:
transmit a further policy to the second system,
wherein the further policy defines a duration that the transformed data received by the second system may be stored on the second system.
14. A system comprising:
a processor; and
a non-transitory storage medium storing instructions executable on the processor to:
identify a first system hosting data where the data meets a data interest of a second system;
receive a request to communicate data between the first system and a second system;
send a query to a policy interface to access rules that establish policies for sharing data between the first system and the second system, the policies comprising a first policy governing transformation of the data from the first system, and a second policy governing a communication protocol to be used; and
transform, according to the first policy, the data from the first system, and communicate the transformed data to the second system according to the communication protocol of the second policy.
15. The system of claim 14 , wherein the policies comprise a time policy that controls a period of time in which the second system may access the data, and wherein the instructions are executable on the processor to send the time policy to the second system.
16. The system product of claim 14 , wherein the policies comprise a policy that contains security protocol regulations between the first system and the second system.
17. The system of claim 14 , wherein the instructions are executable on the processor to:
transmit a further policy to the second system,
wherein the further policy defines a duration that the transformed data received by the second system may be stored on the second system.