Peer-to-peer delta image dispatch system
In an example embodiment, a solution is used to provide for a highly efficient image dispatch system for container-based in-memory databases in a database-as-a-service framework that provides storage and bandwidth cost savings and introduces advanced features of container techniques. This solution makes best use of the advantages of the concurrent and distributed capabilities of in-memory databases in a database-as-a-service framework. A container operator manager provides operators for various uses, such as the “diff” operator and the “merge” operator. A dispatch manager acts as a centralized service for diff content maintenance, patch publication and management of dispatch worker nodes. A patch worker on each node has the capabilities to execute the specified patching and validation between the images located on each side of the in-memory database in the database-as-a-service framework.
1. A system comprising:
at least one hardware processor; and
a non-transitory computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
receiving, at a dispatch manager in an in-memory database database-as-a-service, an updated version of an earlier version of an in-memory database;
obtaining a first container containing a difference operator, the first container being a first containerized application including a first set of runtime components applied to run first software isolated from an operating system;
executing the difference operator on the updated version of the in-memory database and the earlier version of the in-memory database by running the first container at a centralized service, to produce an image patch representing the difference between the updated version and the earlier version;
obtaining, at the centralized service, a second container containing a merge operator, the second container being a second containerized application including a second set of runtime components applied to run second software isolated from the operating system, the second container being different than the first container; and
passing, via a peer-to-peer network, the image patch and the merge operator from the centralized service to a patch worker who has subscribed to updates for the in-memory database, the patch worker being a software component operating on a separate worker node, to cause the patch worker that has subscribed to the updates for the in-memory database to apply merge operator to perform a merge operation between the earlier version of the in-memory database stored by the patch worker and the image patch producing the updated version of the in-memory database locally accessible by the patch worker without the patch worker having to download the updated version of the in-memory database as a whole.
2. The system of claim 1 , wherein the updated version of the in-memory database is received from a continuous integration/continuous delivery service.
3. The system of claim 1 , wherein the image patch is stored in a repository for retrieval and distribution to any future patch worker subscribing to the updates for the in-memory database.
4. The system of claim 1 , wherein the patch worker further performs validation of the merge operation.
5. The system of claim 4 , wherein the validation includes validating that a database version is correct.
6. The system of claim 4 , wherein the validation includes validating that a system schema for the in-memory database is correct.
7. The system of claim 4 , wherein the validation includes validating that specified key data is defined.
8. A method comprising:
receiving, at a dispatch manager in an in-memory database database-as-a-service, an updated version of an earlier version of an in-memory database;
obtaining a first container containing a difference operator, the first container being a first containerized application including a first set of runtime components applied to run first software isolated from an operating system;
executing the difference operator on the updated version of the in-memory database and the earlier version of the in-memory database by running the first container at a centralized service, to produce an image patch representing the difference between the updated version and the earlier version;
obtaining, at the centralized service, a second container containing a merge operator, the second container being a second containerized application including a second set of runtime components applied to run second software isolated from the operating system, the second container being different than the first container; and
passing, via a peer-to-peer network, the image patch and the merge operator from the centralized service to a patch worker who has subscribed to updates for the in-memory database, the patch worker being a software component operating on a separate worker node, to cause the patch worker that has subscribed to the updates for the in-memory database to apply merge operator to perform a merge operation between the earlier version of the in-memory database stored by the patch worker and the image patch producing the updated version of the in-memory database locally accessible by the patch worker without the patch worker having to download the updated version of the in-memory database as a whole.
9. The method of claim 8 , wherein the updated version of the in-memory database is received from a continuous integration/continuous delivery service.
10. The method of claim 8 , wherein the image patch is stored in a repository for retrieval and distribution to any future patch worker subscribing to the updates for the in-memory database.
11. The method of claim 8 , wherein the patch worker further performs validation of the merge operation.
12. The method of claim 11 , wherein the validation includes validating that a database version is correct.
13. The method of claim 11 , wherein the validation includes validating that a system schema for the in-memory database is correct.
14. The method of claim 11 , wherein the validation includes validating that specified key data is defined.
15. A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, at a dispatch manager in an in-memory database database-as-a-service, an updated version of an earlier version of an in-memory database;
obtaining a first container containing a difference operator, the first container being a first containerized application including a first set of runtime components applied to run first software isolated from an operating system;
executing the difference operator on the updated version of the in-memory database and the earlier version of the in-memory database by running the first container at a centralized service, to produce an image patch representing the difference between the updated version and the earlier version;
obtaining, at the centralized service, a second container containing a merge operator, the second container being a second containerized application including a second set of runtime components applied to run second software isolated from the operating system, the second container being different than the first container; and
passing, via a peer-to-peer network, the image patch and the merge operator from the centralized service to a patch worker who has subscribed to updates for the in-memory database, the patch worker being a software component operating on a separate worker node, to cause the patch worker that has subscribed to the updates for the in-memory database to apply merge operator to perform a merge operation between the earlier version of the in-memory database stored by the patch worker and the image patch producing the updated version of the in-memory database locally accessible by the patch worker without the patch worker having to download the updated version of the in-memory database as a whole.
16. The non-transitory machine-readable medium of claim 15 , wherein the updated version of the in-memory database is received from a continuous integration/continuous delivery service.
17. The non-transitory machine-readable medium of claim 15 , wherein the image patch is stored in a repository for retrieval and distribution to any future path worker subscribing to the updates for the in-memory database.
18. The non-transitory machine-readable medium of claim 15 , wherein the patch worker further performs validation of the merge operation.
19. The non-transitory machine-readable medium of claim 18 , wherein the validation includes validating that a database version is correct.
20. The non-transitory machine-readable medium of claim 18 , wherein the validation includes validating that a system schema for the in-memory database is correct.