Scalability improvement in a system which incrementally updates clients with events that occurred in a cloud-based collaboration platform
Techniques are disclosed for improving scalability in a system which incrementally updates remote clients with events that occurred in a cloud-enabled platform. In one embodiment, a method comprises, in response to an action from a user in the cloud-enabled platform, determining a list of events to update one or more collaborators of the user about the action. The method further comprises separating the list of events into a plurality of sub-lists of events such that each sub-list of events can be stored in a database within a designated amount of time, and further comprises storing the plurality of sub-lists of events into the database to be read by the one or more collaborators. Among other advantages, embodiments disclosed herein provide enhancement in scalability, robustness and availability for cloud-based collaboration platforms with large numbers of collaborators by incorporating mechanisms to divide-and-conquer the workload of event updates in such platforms.
1. A method for updating remote clients with actions that occurred in a cloud-enabled platform, the method comprising:
in response to an action from a user in the cloud-enabled platform, determining a list of events to update one or more collaborators of the user about the action;
separating the list of events into a plurality of sub-lists of events by decomposing the list of events;
determining based on evaluation of one or more criteria associated with the database, a number of events to include in each sub-list of events from the plurality of sub-lists of events;
writing each sub-list of events into queues that are stored in a database, wherein the writing occurs within a designated time window, wherein the designated time window specifies a time limit for writing each sub-list of events into the queues; and
upon detecting failure to write a sub-list of events into one or more queues, updating the designated time window for the sub-list of events that failed to be written, wherein the sub-list of events that failed to be written is included in the plurality of sub-lists of events.
2. The method of claim 1 , wherein the one or more criteria includes a throughput of the database.
3. The method of claim 1 , further comprising:
identifying a number of sub-lists of events that fails to be stored in the database within the designated time window.
4. The method of claim 1 , wherein the one or more criteria includes a performance of a network connection of the database.
5. The method of claim 1 , wherein determining the list of events to update the one or more collaborators of the user about the action is based on requesting a business logic to return a manifest of the one or more collaborators associated with a folder where the action from the user takes place.
6. The method of claim 1 , wherein the plurality of sub-lists are stored as rows in a table before being stored into the database, and wherein each row represents one sub-list and has an collaborator identification field that specifies which of the one or more collaborators are to be updated about the action.
7. The method of claim 6 , further comprising:
writing a timestamp for a respective row upon completion of storage of a respective sub-list into the database.
8. The method of claim 1 , wherein the remote clients of the one or more collaborators are able to update based on the database to reflect the action.
9. The method of claim 8 , wherein a respective remote client in the remote clients update only the action without updating other actions which have previously been updated at the respective remote client.
10. The method of claim 8 , wherein a respective remote client in the remote clients sends a query to the database for synchronization of the updates specific to an associated collaborator.
11. A system for incrementally updating remote clients with actions that occurred in a cloud-based environment, the system comprising:
a processor; and
a memory coupled to the processor and storing a plurality of instructions which, when executed by the processor, cause the processor to:
in response to an action from a user in the cloud-enabled platform, determine a list of events to update one or more collaborators of the user about the action;
separate the list of events into a plurality of sub-lists of events by decomposing the list of events;
determine, based on evaluation of one or more criteria associated with the database, a number of events to include in each sub-list of events from the plurality of sub-lists of events;
write each sub-list of events into queues that are stored in a database, wherein the write occurs within a designated time window, wherein the designated time window specifies a time limit for writing each sub-list of events into the queues; and
upon detecting failure to write a sub-list of events into one or more queues, update the designated time window for the sub-list of events that failed to be written, wherein the sub-list of events that failed to be written is included in the plurality of sub-lists of events.
12. The system of claim 11 , wherein the one or more criteria includes a throughput of the database.
13. The system of claim 11 , wherein the processor is further caused to:
identify a number of sub-lists of events that fails to be stored in the database within the designated time window.
14. The system of claim 11 , wherein the one or more criteria includes a performance of a network connection of the database.
15. The system of claim 11 , wherein the processor is further caused to, in the determining the list of events to update the one or more collaborators of the user about the action, request a business logic to return a manifest of the one or more collaborators associated with a folder where the action from the user takes place.
16. The system of claim 11 , wherein the plurality of sub-lists are stored as rows in a table before being stored into the database, and wherein each row represents one sub-list and has an collaborator identification field that specifies which of the one or more collaborators are to be updated about the action.
17. The system of claim 16 , wherein the processor is further caused to:
write a timestamp for a respective row upon completion of storage of a respective sub-list into the database.
18. A non-transitory machine-readable storage medium having stored thereon instructions which, when executed by a processor, cause the processor to:
in response to an action from a user in a cloud-enabled platform, determine a list of events to update one or more collaborators of the user about the action;
separate the list of events into a plurality of sub-lists of events by decomposing the list of events;
determine, based on evaluation of one or more criteria associated with the database, a number of events to include in each sub-list of events from the plurality of sub-lists of events;
write each sub-list of events into queues that are stored in a database, wherein the write occurs within a designated time window, wherein the designated time window specifies a time limit for writing each sub-list of events into the queues; and
upon detecting failure to write a sub-list of events into one or more queues, update the designated time window for the sub-list of events that failed to be written, wherein the sub-list of events that failed to be written is included in the plurality of sub-list of events.
19. The storage medium of claim 18 , wherein the one or more criteria includes a throughput of the database.
20. The storage medium of claim 18 , wherein the processor is further caused to:
identify a number of sub-lists of events that fails to be stored in the database within the designated time window.
21. The storage medium of claim 18 , wherein the one or more criteria includes a performance of a network connection of the database.
22. The storage medium of claim 18 , wherein the processor is further caused to, in the determining the list of events to update the one or more collaborators of the user about the action, request a business logic to return a manifest of the one or more collaborators associated with a folder where the action from the user takes place.
23. The storage medium of claim 18 , wherein the plurality of sub-lists are stored as rows in a table before being stored into the database, and wherein each row represents one sub-list and has an collaborator identification field that specifies which of the one or more collaborators are to be updated about the action.
24. The storage medium of claim 23 , wherein the processor is further caused to:
write a timestamp for a respective row upon completion of storage of a respective sub-list into the database.
25. A system, comprising a processor, wherein the processor is configured for:
in response to an action from a user in a cloud-enabled platform, determining a list of events to update one or more collaborators of the user about the action;
separating the list of events into a plurality of sub-lists of events by decomposing the list of events;
determining, based on evaluation of one or more criteria associated with the database, a number of events to include in each sub-list of events from the plurality of sub-lists of events;
writing each writing each sub-list of events into queues that are stored in a database, wherein the writing occurs within a designated time window, wherein the designated time window specifies a time limit for writing each sub-list of events into the queues; and
upon detecting failure to write a sub-list of events into one or more queues, updating the designated time window for the sub-list of events that failed to be written, wherein the sub-list of events that failed to be written is included in the plurality of sub-list of events.