Agent message delivery fairness
Apparatus and methods are disclosed for generating, sending, and receiving messages in a networked environment using autonomous (or semi-autonomous) agents. In one example of the disclosed technology, a method of controlling message flow in a computer network comprising a plurality of agents, agent data consumers, and an agent message bridge configured to send messages by receiving a set of messages, at least some of the messages including a message type, queuing the set of messages in a spooler that includes an indication of the respective message type for each of the messages, receive an indication that sending some of the messages queued in the spooler should be delayed for one or more indicated message types, and sending at least one of the messages to a selected one or more of the agent data consumers, the sent messages not being of the indicated message types.
1. A method of collecting data from an agent executing on a host computer, the host computer being connected to one or more agent data consumers via a network connection, the method comprising:
with the agent, collecting host data, the collecting occurring whether or not the agent can currently send data via the network connection, the host computer executing one or more plugins to collect the host data;
when the agent cannot send the data via the network connection, storing at least a portion of the collected host data in a spooler at the host computer, the plugins adjusting the rate of collecting host data based at least in part on the indicated spooler lag;
when the agent can send the data via the network connection, sending at least a portion of the spooled host data to at least one of the agent data consumers, the agent sending data to the plugins indicating lag of the spooler;
receiving a message from an agent bridge indicating a message type to send to at least one of the agent data consumers; and
initiating sending of data for the indicated message type.
2. The method of claim 1 , wherein the message is a first message and the message type is a first message type, the method further comprising:
receiving a second message from an agent bridge indicating a second message type to not send to at least one of the agent data consumers; and
based on the receiving the second message, storing data for messages of the second message type in a spooler until receiving a third message indicating the spooled messages of the second type are to be sent.
3. The method of claim 2 , wherein messages of the first message type are stored in a first spool of the spooler and where message of the second message type are stored in a second spool of the spooler.
4. The method of claim 1 , wherein the rate at which data is spooled and/or the rate at which the spooled data is sent to the one or more agent data consumers is based at least in part on currently-available resources of the host computer.
5. The method of claim 1 , further comprising receiving a UUID for the at least one of the agent data consumers, wherein the agent sends the data including an indication of the UUID for routing the data to the at least one of the agent data consumers.
6. The method of claim 1 , wherein when the agent cannot send the data via the network connection is due to network disconnections.
7. The method of claim 1 , wherein when the agent cannot send the data via the network connection is due to intentional or unintentional loss of network connectivity to the agent bridge.
8. The method of claim 1 , wherein when the agent cannot send the data via the network connection, the agent bridge sends an agent disconnect message to at least one of the agent data consumers.
9. The method of claim 8 , wherein when the agent can send the data via the network connection after when the agent cannot send the data, the agent bridge sends an agent connect message to at least one of the agent data consumers.
10. A method of collecting data from an agent executing on a host computer, the host computer being connected to one or more agent data consumers via a network connection, the method comprising:
with the agent, collecting host data, the collecting occurring whether or not the agent can currently send data via the network connection; the host data being collected by one or more plugins executable on the host computer when the agent cannot send data via the network connection;
when the agent cannot send the data via the network connection, storing at least a portion of the collected host data in a spooler at the host computer;
when the agent can send the data via the network connection, sending at least a portion of the spooled host data to at least one of the agent data consumers the spooled host data being sent to the at least one of the agent data consumers via a bridge executing on an agent platform server;
receiving a message from an agent bridge indicating a message type to send to at least one of the agent data consumers; and
initiating sending of data for the indicated message type.
11. The method of claim 10 , wherein when the agent cannot send the data via the network connection, the agent bridge sends an agent disconnect message to at least one of the agent data consumers.
12. The method of claim 11 , wherein when the agent can send the data via the network connection after when the agent cannot send the data, the agent bridge sends an agent connect message to at least one of the agent data consumers.
13. One or more computer-readable storage media storing computer-executable instructions, which when executed by a processor, cause a system to perform a method of collecting data from an agent executing on a host computer, the host computer being connected to one or more agent data consumers via a network connection, the computer-executable instruction comprising:
instructions that cause the agent to collect host data, the collecting occurring whether or not the agent can currently send data via the network connection; the host data being collected by one or more plugins executable on the host computer when the agent cannot send data via the network connection;
instructions that cause the agent to, when the agent cannot send the data via the network connection, storing at least a portion of the collected host data in a spooler at the host computer;
instructions that cause the agent to, when the agent can send the data via the network connection, sending at least a portion of the spooled host data to at least one of the agent data consumers the spooled host data being sent to the at least one of the agent data consumers via a bridge executing on an agent platform server;
instructions that cause the agent to receive a message from an agent bridge indicating a message type to send to at least one of the agent data consumers; and
instructions that cause the agent to initiate sending of data for the indicated message type.
14. The computer-readable storage media of claim 13 , further comprising instructions that when the agent cannot send the data via the network connection, cause the agent bridge to send an agent disconnect message to at least one of the agent data consumers.
15. The computer-readable storage media of claim 14 , further comprising instructions that when the agent can send the data via the network connection after when the agent cannot send the data, cause the agent bridge to send an agent connect message to at least one of the agent data consumers.
16. A system, comprising:
a computer network providing a network connection;
one or more agent data consumers;
one or more agent platform servers, each of the agent platform servers including an agent bridge, the agent bridge being in communication with the agent platform servers via the network connection and being further configured to send a message including a message type indicating a message type for at least one of the agent data consumers to send at least one respective agent platform servers;
computing hosts comprising a processor and memory executing the one or more agents, the agents being configured to:
execute on the computing hosts separate from the agent platform servers,
collect host data whether or not the agent can currently send data to the agent platform servers,
when the agent cannot send the data via the network connection, storing at least a portion of the collected host data in a spooler at the agent's respective computing host, and
when the agent can send the data via the network connection, sending at least a portion of the spooled host data to at least one of the agent data consumers; and
an affinity service, the affinity service being configured to map each of the agents to a respective one of the data consumers.
17. The system of claim 16 , wherein the affinity service mapping is based at least in part on one or more of the following: a geographic location of one or more of the agents and/or the agent data consumers; topology of a network carrying communication between the agent data consumers, agent platform servers, and/or agent computing hosts; a functional role of one of the agents and/or one of the agent data consumers; a version of an agent; and/or a version of an agent data consumer.