Method for optimizing the management of a flow of data
A method is used for managing a flow of data in at least one database, wherein said database is configured with at least two data models of data storage. In said method, during a first period of time, a first data flow portion is received in a computer, and the first data flow portion is then stored in a first data pool of the database according to the first data model. Then, after the first period of time, a transformation is made on the first portion of the data and the transformed first data is assigned to a second data model, and the first data flow portion is then transferred from the first data pool to a second data pool. The process is repeated with at least a second data flow portion and can be extended to more transformations, and, thus, to more data models, and more data pools accordingly.
1. A computer-implemented method for managing a flow of data in at least one database, wherein the flow of data comprises at least a first data flow portion, a second data flow portion and a third data flow portion, and said at least one database is configured with at least a first, a second and a third data models of data storage, and wherein the method comprises performing the following steps:
a) receiving, during a first period of time, the first data flow portion in a computer;
b) storing said first data flow portion in a first data pool of the at least one database according to the first data model;
c) after the first period of time, cascading the first data flow portion from the first data pool to a second data pool in the at least one database and assigning the first data flow portion to the second data model;
d) repeating steps a)-c) for at least the second data flow portion;
e) repeating steps a)-c) for at least the third data flow portion; and
f) after a second period of time, cascading the data from the second data pool to a third data pool in the at least one database, and assigning said data to the third data model.
2. The method according to claim 1 , where at least the first, the second or the third data model comprises a key-valued model, a short-column model, or a long-column model.
3. The method according to claim 2 , where the first data model comprises a key-valued model.
4. The method according to claim 2 , where the second data model comprises a short-column model.
5. The method according to claim 2 , wherein the third data model comprises a long-column model.
6. The method according to claim 1 , wherein the first period of time is smaller than the second period of time.
7. The method according to claim 1 , where the flow of data is provided by an IoT infrastructure, one or more sensors.
8. The method according to claim 1 , wherein the first, the second or the third data pool is configured with at least two polyglot abstraction layers.
9. The method according to claim 1 , comprising at least a first and a second database, wherein the first data pool is stored in the first database, and the second data pool is stored in the second database.
10. A computer comprising hardware and software adapted to perform a method according to claim 1 .
11. An IoT system comprising a computer according to claim 10 , communicatively connected to a plurality of IoT devices adapted to generate at least a flow of data and to send said flow of data to the computer.
12. The method according to claim 1 , where the first, the second, and/or the third data model comprise in-disk organizations of the data.
13. A computer program product comprising one or more computer-readable hardware storage devices having thereon computer-executable instructions that are structured such that, when executed by one or more processors of a local computing system, cause the computing system to perform a method for managing a flow of data in at least one database, the method comprising:
receiving, during a first period of time, a first data flow portion;
storing said first data flow portion in a first data pool of at least one database according to a first data model;
after the first period of time, cascading the first data flow portion from the first data pool to a second data pool in the at least one database and assigning the first data flow portion to a second data model;
repeating steps a)-c) for at least a second data flow portion;
repeating steps a)-c) for at least a third data flow portion; and
after a second period of time, cascading the data from the second data pool to a third data pool in the at least one database, and assigning said data to a third data model.
14. A non-transitory computer readable medium containing instructions that when executed by at least one processor, cause the at least one processor to perform a method for managing a flow of data in at least one database, the method comprising:
receiving, during a first period of time, a first data flow portion;
storing said first data flow portion in a first data pool of at least one database according to a first data model;
after the first period of time, cascading the first data flow portion from the first data pool to a second data pool in the at least one database and assigning the first data flow portion to a second data model;
repeating steps a)-c) for at least a second data flow portion;
repeating steps a)-c) for at least a third data flow portion; and
after a second period of time, cascading the data from the second data pool to a third data pool in the at least one database, and assigning said data to a third data model.