Message consumer orchestration framework
In a decoupled messaging system, a producer device sends produced messages to a message fabric. The message fabric either stores the received message in designated queues, from which consumer devices may retrieve the messages, or broadcasts the messages to consumer devices that subscribe to topics assigned to the messages. A message consumption orchestrator service gathers capabilities information for the message producer, the message fabric, and the message consumer, and configures resources allocated for the message consumer to more fully optimize message throughput. Further, the message consumption orchestrator identifies patterns in changes of the rate of produced messages, i.e., through analysis of historical data and/or received information. The message consumption orchestrator configures resources for the message consumer to account for predicted need based on the identified patterns.
1. A method comprising:
based on historical information, identifying a pattern in timing of changes in a rate of message flow between a message producer and a message fabric;
based, at least in part, on the pattern, predicting a future change to a flow of messages from the message producer through the message fabric to a message consumer;
in response to predicting the future change to the flow of messages, automatically scheduling a future adjustment of consumer resources allocated to the message consumer;
wherein the future adjustment is scheduled to occur at a scheduled adjustment time; and
in response to reaching the scheduled adjustment time, and based, at least in part, on the predicted change, automatically adjusting consumer resources allocated to the message consumer;
receiving at least a portion of the historical information from a third party information source;
wherein the historical information comprises information identifying two or more critical dates of a certain type;
wherein a first critical date of the two or more critical dates occurred in the past;
wherein predicting a future change to a flow of messages from the message producer through the message fabric to the message consumer comprises:
determining that, on the first critical date, the flow of messages had a particular peak rate, and
based, at least in part, on the type of the first critical date being the same as the type of a second critical date, of the two or more critical dates, predicting that the second critical date of the two or more critical dates is the date of the future change to the flow of messages; and
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , further comprising gathering at least a portion of the historical information from one or more of the message producer, the message fabric, or the message consumer.
3. One or more non-transitory computer readable media storing one or more sequences of instructions which, when executed by one or more processors, cause:
based on historical information, identifying a pattern in timing of changes in a rate of message flow between a message producer and a message fabric;
based, at least in part, on the pattern, predicting a future change to a flow of messages from the message producer through the message fabric to a message consumer;
in response to predicting the future change to the flow of messages, automatically scheduling a future adjustment of consumer resources allocated to the message consumer;
wherein the future adjustment is scheduled to occur at a scheduled adjustment time; and
in response to reaching the scheduled adjustment time, and based, at least in part, on the predicted change, automatically adjusting consumer resources allocated to the message consumer;
receiving at least a portion of the historical information from a third party information source;
wherein the historical information comprises information identifying two or more critical dates of a certain type;
wherein a first critical date of the two or more critical dates occurred in the past; and
wherein predicting a future change to a flow of messages from the message producer through the message fabric to the message consumer comprises:
determining that, on the first critical date, the flow of messages had a particular peak rate, and
based, at least in part, on the type of the first critical date being the same as the type of a second critical date, of the two or more critical dates, predicting that the second critical date of the two or more critical dates is the date of the future change to the flow of messages.
4. The one or more non-transitory computer readable media of claim 3 , wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause gathering at least a portion of the historical information from one or more of the message producer, the message fabric, or the message consumer.