Data stream processor and method to throttle consumption of message data in a distributed computing system

1. A computer-implemented method comprising:

under control of a computing system comprising one or more processors configured to execute specific instructions,

receiving, from one or more computing devices, message data to access a pool of data, wherein a message represented by the message data corresponds to an application programming interface call from a computing device of the one or more computing devices to access the pool of data;

classifying a subset of messages into a classified subset of messages;

storing the classified subset of messages to a partition of memory, the memory including a plurality of independently-accessible partitions, wherein the partition is from the plurality of independently-accessible partitions;

accessing a data stream including the classified subset of messages;

determining a stream characteristic for the data stream;

receiving additional message data associated with the data stream, wherein the additional message data represents a plurality of additional messages to access the pool of data;

incrementing a value of the stream characteristic for each additional message of the plurality of additional messages;

reducing the value of the stream characteristic by a quantity per unit of time;

determining that the value of the stream characteristic satisfies a threshold;

generating throttle data; and

throttling handling of subsequent message data received from the computing device based at least in part on the throttle data.

2. The computer-implemented method of claim 1 , further comprising monitoring the value of the stream characteristic concurrently with receiving at least a portion of the additional message data.

3. The computer-implemented method of claim 1 , wherein accessing the data stream comprises accessing directly data representing the stream characteristic from the data stream.

4. The computer-implemented method of claim 1 , wherein generating the throttle data comprises embedding in the throttle data:

an instruction to suspend processing of the subsequent message data received from the computing device; and

data representing a time at which the processing of the subsequent message data received from the computing device is re-enabled.

5. The computer-implemented method of claim 1 , further comprising determining the threshold as a value representing a threshold number of calls by the computing device per unit of time.

6. The computer-implemented method of claim 5 , further comprising

determining data representing an identifier of the computer device;

transmitting an instruction to suspend processing of the subsequent message data received from the computing device for a duration of time.

7. The computer-implemented method of claim 5 , further comprising:

accessing the value of the threshold stored in memory; and

implementing at least a portion of a data stream processor in memory.

8. The computer-implemented method of claim 1 , further comprising determining the threshold as a value representing a number of calls associated with a type of message per unit of time.

9. The computer-implemented method of claim 8 , further comprising determining the type of message subject to throttling wherein determining that the value of the stream characteristic satisfies the threshold comprises determining that a quantity of calls associated with the type of message exceeds the threshold.

10. The computer-implemented method of claim 1 , further comprising adjusting a value of the threshold from a first threshold value to a second threshold value based at least in part on the stream characteristic.

11. A system comprising:

a data store; and

one or more computing devices in communication with the data store and configured to at least:

receive message data representing one or more programmatic calls to access a pool of data;

classify a subset of the message data into a classified subset of message data;

store the classified subset of message data in the data store;

determine a stream characteristic for a data stream, wherein the data stream includes the classified subset of message data;

receive additional message data associated with the data stream, wherein the additional message data represents a plurality of additional programmatic calls to access the pool of data;

increment a value of the stream characteristic for each additional programmatic call of the plurality of programmatic calls;

reduce the value of the stream characteristic by a quantity per unit of time;

determine that the value of the stream characteristic exceeds a threshold; and

generate throttle data representing an instruction to throttle handling of subsequent message data received from a remote computing device from which a programmatic call of the plurality of additional programmatic calls originates.

12. The system of claim 11 , further comprising monitoring the value of the stream characteristic concurrently with receiving at least a portion of the additional message data.

13. The system of claim 11 , wherein the one or more computing devices access data representing the stream characteristic directly from the data stream.

14. The system of claim 11 , wherein the throttle data further represents a time at which the processing of subsequent message data received from the remote computing device is re-enabled.

15. The system of claim 11 , wherein the one or more computing devices are further configured to at least:

determine data representing an identifier of the remote computer device; and

transmit an instruction to cease handling of subsequent message data received from the remote computing device for a duration of time.

16. A computer-implemented method comprising:

under control of a computing system comprising one or more processors configured to execute specific instructions,

receiving message data representing one or more programmatic calls to access a pool of data;

classifying a subset of the message data into a classified subset of message data;

storing the classified subset of message data to a log file, the log file including a number of independently-accessible partitions;

accessing a data stream including the classified subset of message data;

determining a stream characteristic of the data stream;

receiving additional message data associated with the data stream, wherein the additional message data represents a plurality of additional programmatic calls to access the pool of data;

incrementing a value of the stream characteristic for each additional programmatic call of the plurality of programmatic calls;

reducing the value of the stream characteristic by a quantity per unit of time;

determining that the value of the stream characteristic exceeds a threshold; and

generating throttle data representing an instruction to throttle handling of subsequent message data received from a remote computing device from which a programmatic call of the plurality of additional programmatic calls originates.

17. The computer-implemented method of claim 16 , further comprising monitoring the value of the stream characteristic concurrently with receiving at least a portion of the additional message data.

18. The computer-implemented method of claim 16 , wherein accessing the data stream comprises accessing directly data representing the stream characteristic from the data stream.

19. The computer-implemented method of claim 16 , wherein generating the throttle data comprises embedding in the throttle data:

an instruction to suspend processing the subsequent message data received from the remote computing device; and

data representing a time at which processing the subsequent message data received from the remote computing device is re-enabled.

20. The computer-implemented method of claim 16 , further comprising:

determining data representing an identifier of the remote computer device; and

transmitting an instruction to cease handling of subsequent message data received from the remote computing device for a duration of time.