Method for optimized canary deployments for improved customer experience
Described embodiments may provide deployment of updates to multiple entities. A canary deployment manager may receive a request to deploy an update in canary deployments. The canary deployment manager may determine, for each entity of a plurality of entities, a support score based on support issues experienced by each entity. The canary deployment manager may determine, for each entity, a monitoring score based on at least one performance or availability issue experienced by each entity. The canary deployment manager may generate, for each entity, based on the support score of each entity and the monitoring score of each entity, a canary score used to identify a canary deployment to which to assign each entity. The canary deployment manager may select a subset of the plurality of entities to assign to a first canary wave of the canary deployments based on the respective canary scores of each of the subset.
1. A system for deploying updates to multiple entities, comprising:
a device including a canary deployment manager, the canary deployment manager configured to:
receive a request to deploy an update in a plurality of canary deployments;
determine, for each entity of a plurality of entities, a support score based on support issues experienced by each entity;
determine, for each entity of the plurality of entities, a monitoring score based on at least one performance or availability issue experienced by each entity;
generate, for each entity of the plurality of entities, based on the support score of each entity and the monitoring score of each entity, a canary score used to identify a canary deployment to which to assign each entity; and
select a subset of the plurality of entities to assign to a first canary wave of the plurality of canary deployments based on the respective canary scores of each of the subset of entities.
2. The system of claim 1 , wherein the canary deployment manager is configured to determine, for each entity, a number of customer support issues logged by the entity or a severity level of the customer support issues logged by the entity; and
wherein to determine, for each entity of a plurality of entities, the support score based on support issues experienced by each entity, the canary deployment manager is configured to determine, for each entity, the support score based on the number of customer support issues logged by the entity or the severity level of the customer support issues logged by the entity.
3. The system of claim 1 , wherein the canary deployment manager is configured to determine, for each entity, a length of time since the support issue was experienced by each entity; and
wherein to determine, for each entity of a plurality of entities, the support score based on support issues experienced by each entity, the canary deployment manager is configured to determine, for each entity, the support score based on the length of the time since the support issue was experienced by each entity.
4. The system of claim 1 , wherein the canary deployment manager is configured to determine, for each entity, a number of performance degradation issues experienced by the entity that are monitored by the device or a severity level of the performance degradation issues experienced by the entity that are monitored by the device; and
wherein to determine, for each entity of the plurality of entities, the monitoring score based on the at least one performance or availability issue experienced by each entity, the canary deployment manager is configured to determine, for each entity, the monitoring score based on the number of performance degradation issues experienced by the entity that are monitored by the device or the severity level of the performance degradation issues experienced by the entity that are monitored by the device.
5. The system of claim 1 , wherein the canary deployment manager is configured to determine, for each entity, a length of time since the at least one performance or availability issue was experienced by each entity; and
wherein to determine, by the device, for each entity of the plurality of entities, the monitoring score based on the at least one performance or availability issue experienced by each entity, the canary deployment manager is configured to determine, for each entity, the monitoring score based on the length of the time since the at least one performance or availability issue was experienced by each entity.
6. The system of claim 1 , wherein the canary deployment manager is configured to determine, for each entity of a plurality of entities, a complexity score determined based on a complexity of a schema or environment of each entity; and wherein to generate, for each entity, the canary score, the canary deployment manager is configured to generate the canary score of each entity based on the complexity score of each entity.
7. The system of claim 6 , wherein the canary deployment manager is configured to determine, for each entity of a plurality of entities, a scale score determined based on a number of accounts managed by each entity; and wherein to generate, for each entity, the canary score, the canary deployment manager is configured to generate the canary score of each entity based on the scale score of each entity.
8. The system of claim 1 , wherein to determine, for each entity of the plurality of entities, the monitoring score based on at least one performance or availability issue experienced by each entity, a monitor of the device is configured to record the at least one performance or availability issue experienced by each entity.
9. The system of claim 1 , wherein to select a subset of the plurality of entities to assign to a first canary deployment of the plurality of canary deployments based on the respective canary scores of each of the subset of entities, the canary deployment manager is configured to select each entity of the subset based on i) the respective canary score of each entity relative to a predetermined threshold score or ii) the respective canary score of each entity relative to respective canary scores of a remainder of the plurality of entities.
10. The system of claim 1 , wherein the subset of the plurality of entities to assign to a first canary deployment of the plurality of canary deployments is a first subset; and wherein the canary deployment manager is configured to select a second subset of the plurality of entities to assign to a second canary deployment of the plurality of canary deployments based on the respective canary scores of each of the second subset of entities.
11. A method for deploying updates to multiple entities, comprising:
receiving, by a device, a request to deploy an update in a plurality of canary deployments;
determining, by the device, for each entity of a plurality of entities, a support score based on support issues experienced by each entity;
determining, by the device, for each entity of the plurality of entities, a monitoring score based on at least one performance or availability issue experienced by each entity;
generating, by the device, for each entity of the plurality of entities, based on the support score of each entity and the monitoring score of each entity, a canary score used to identify a canary deployment to which to assign each entity; and
selecting, by the device, a subset of the plurality of entities to assign to a first canary deployment of the plurality of canary deployments based on the respective canary scores of each of the subset of entities.
12. The method of claim 11 , comprises determining, by the device, for each entity, a number of customer support issues logged by the entity or a severity level of the customer support issues logged by the entity; and
wherein determining, for each entity of a plurality of entities, the support score based on support issues experienced by each entity comprises determining, for each entity, the support score based on the number of customer support issues logged by the entity or the severity level of the customer support issues logged by the entity.
13. The method of claim 11 , comprises determining, by the device, for each entity, a length of time since the support issue was experienced by each entity; and
wherein determining, for each entity of a plurality of entities, the support score based on support issues experienced by each entity comprises determining, for each entity, the support score based on the length of the time since the support issue was experienced by each entity.
14. The method of claim 11 , comprises determining, by the device, for each entity, a number of performance degradation issues experienced by the entity that are monitored by the device or a severity level of the performance degradation issues experienced by the entity that are monitored by the device; and
wherein determining, by the device, for each entity of the plurality of entities, the monitoring score based on the at least one performance or availability issue experienced by each entity comprises determining, for each entity, the monitoring score based on the number of performance degradation issues experienced by the entity that are monitored by the device or the severity level of the performance degradation issues experienced by the entity that are monitored by the device.
15. The method of claim 11 , comprising determining, by the device, for each entity, a length of time since the at least one performance or availability issue was experienced by each entity; and
wherein determining, for each entity of the plurality of entities, the monitoring score based on the at least one performance or availability issue experienced by each entity comprises determining, for each entity, the monitoring score based on the length of the time since the at least one performance or availability issue was experienced by each entity.
16. The method of claim 11 , comprising determining, by the device, for each entity of a plurality of entities, a complexity score determined based on a complexity of a schema or environment of each entity; and wherein generating, for each entity, the canary score comprises generating the canary score of each entity based on the complexity score of each entity.
17. The method of claim 11 , comprising determining, by the device, for each entity of a plurality of entities, a scale score determined based on a number of accounts managed by each entity; and wherein generating, for each entity, the canary score comprises generating the canary score of each entity based on the scale score of each entity.
18. The method of claim 11 , wherein determining, for each entity of the plurality of entities, the monitoring score based on at least one performance or availability issue experienced by each entity comprises recording, by a monitor of the device, the at least one performance or availability issue experienced by each entity.
19. The method of claim 11 , wherein selecting, by the device, a subset of the plurality of entities to assign to a first canary deployment of the plurality of canary deployments based on the respective canary scores of each of the subset of entities comprises selecting, by the device, each entity of the subset based on i) the respective canary score of each entity relative to a predetermined threshold score or ii) the respective canary score of each entity relative to respective canary scores of a remainder of the plurality of entities.
20. The method of claim 11 , wherein the subset of the plurality of entities to assign to a first canary deployment of the plurality of canary deployments is a first subset; and further comprising selecting, by the device, a second subset of the plurality of entities to assign to a second canary deployment of the plurality of canary deployments based on the respective canary scores of each of the second subset of entities.