Data stream processor and method to counteract anomalies in data streams transiting a distributed computing system
Various embodiments relate generally to electrical and electronic hardware, computer software and systems for controlling a data stream processor configured to detect and/or resolve anomalies in data streams including message data. In particular, a system, a device and a method may be configured to access multiple data streams and to detect an anomaly, in real-time or in substantially real-time, that is associated with at least one of the data streams accessed by a data stream processor. In some examples, a method can include one or more of receiving message data to facilitate a computerized rental of property, classifying subset of messages, fetching the classified messages to form multiple data streams, accessing the data stream to indemnity a stream characteristic, detecting an anomaly based on an identified stream characteristic, and generating anomaly resolution data to counteract the detected anomaly.
1. A computer-implemented method comprising:
as performed by a computing system comprising one or more computer processors configured to execute specific instructions,
receiving message data from a plurality of computing devices regarding rental of real property, wherein message data received from a first computing device of the plurality of computing devices represents a message associated with a programmatic call generated by a user interface portion of a plurality of user interface portions concurrently displayed on the first computing device;
classifying the message data into classified subsets of messages, wherein a classified subset of messages of the classified subsets of messages is classified based at least partly on a user interface portion from which individual messages in the classified subset of messages are generated;
generating a plurality of data streams, wherein a data stream of the plurality of data streams comprises messages from the classified subset of messages;
identifying a stream characteristic of the data stream based at least partly on data associated with a message of the classified subset of messages;
detecting an anomaly associated with the data stream based at least partly on parametric data associated with the classified subset of messages and the stream characteristic; and
generating anomaly resolution data regarding the anomaly.
2. The computer-implemented method of claim 1 , wherein identifying the stream characteristic is performed in real-time as data in the data stream transits between a first computing device storing the classified subsets of messages and a second computing device configured to consume data in the data stream.
3. The computer-implemented method of claim 1 , further comprising accessing data representing the stream characteristic from the data stream.
4. The computer-implemented method of claim 1 , further comprising:
identifying a plurality of parametric values using (1) the stream characteristic and (2) a plurality of other stream characteristics;
identifying a correlated subset of the plurality of parametric values based at least partly on the stream characteristic and the plurality of other stream characteristics;
classifying the correlated subset of the plurality parametric values as non-anomalous; and
generating an anomaly threshold for the data stream based at least partly on the correlated subset of the plurality of parametric values.
5. The computer-implemented method of claim 4 , wherein identifying the plurality of parametric values comprises:
characterizing one or more of: computer identifiers, a transit time, or a message type;
classifying, as non-anomalous, parametric values of the one or more of: computer identifiers, the transit time, or the message type; and
setting the anomaly threshold for the data stream based at least partly on the parametric values of the one or more of: computer identifiers, the transit time, or the message type.
6. The computer-implemented method of claim 4 , further comprising:
executing instructions on a training computing device to perform machine learning; and
classifying the correlated subset of the plurality of parametric values based at least partly on the machine learning.
7. The computer-implemented method of claim 1 , wherein generating the anomaly resolution data comprises generating action data comprising at least one of: alert data or corrective action data.
8. The computer-implemented method of claim 7 , wherein generating the action data comprises:
generating resolution data configured to cause the stream characteristic to transition to a non-anomalous value.
9. The computer-implemented method of claim 7 , wherein generating the action data comprises:
generating real-time alert data representing an indication that the anomaly is detected; and
transmitting the real-time alert data in an electronic message including an SMS format.
10. The computer-implemented method of claim 1 , further comprising adjusting a parametric value in a non-compliant state to transition to a compliant state.
11. The computer-implemented method of claim 1 , further comprising transmitting, to the first computing device, a network resource comprising instructions for display of the plurality of user interface portions.
12. The computer-implemented method of claim 1 , wherein the user interface portion comprises a predefined display region of a user interface, the predefined display region comprising a plurality of interactive elements configured to generate programmatic calls.
13. A system comprising:
one or more producer computing devices, wherein a producer computing device of the one or more producer computing devices is configured to receive message data from one or more computing devices, wherein message data received from a first computing device of the one or more computing devices comprises a programmatic call relating to effecting rental of real property, generated by a user interface portion of a plurality of user interface portions concurrently displayed on the first computing device;
a plurality of broker computing devices, wherein a broker computing device of the plurality of broker computing devices is configured to:
classify at least a portion of the message data into a classified subset of messages based at least partly on a user interface portion from which individual messages in the classified subset of messages were generated; and
append the classified subset of messages to a log file;
one or more consumer computing devices, wherein a consumer computing device of the one or more consumer computing devices is configured to:
subscribe to the log file; and
determine, based at least partly on the message data, data regarding a subset of rental properties to be presented by a computing device; and
one or more data-stream computing devices, wherein a data-stream computing device of the one or more data-stream computing devices is configured to:
access a plurality of data streams between the plurality of broker computing devices and the one or more consumer computing devices;
identify a data stream of the plurality of data streams;
detect an anomaly associated with the data stream based at least partly on a stream characteristic of the data stream; and
generate anomaly resolution data regarding the anomaly.
14. The system of claim 13 , wherein the data-stream computing device is further configured to identify the stream characteristic in real-time as data in the data stream transits between a first computing device storing the classified subset of messages and a second computing device configured to consume data in the data stream.
15. The system of claim 13 , wherein the data-stream computing device is further configured to:
identify parametric values based at least partly on (1) the stream characteristic and (2) a plurality of other stream characteristics;
identify correlated subsets of the parametric values based at least partly on the stream characteristic and the plurality of other stream characteristics to form correlated parametric values;
classify a first subset of the correlated subsets of the parametric values as non-anomalous; and
generate an anomaly threshold for the data stream based at least partly on the first subset.
16. The system of claim 13 , wherein the data-stream computing device is further configured to generate at least one of alert data or corrective action data.
17. The system of claim 13 , wherein the user interface portion is configured to generate the programmatic call responsive to activation of an interactive element of the user interface portion.
18. Non-transitory computer-readable storage storing executable instructions that configure a computing system to perform a process comprising:
receiving message data from one or more computing devices, wherein message data received from a first computing device of the one or more computing devices is associated with a programmatic call relating to transacting rental of real property, generated by a user interface portion of a plurality of user interface portions concurrently displayed on the first computing device;
classifying the message data into classified subsets of messages, wherein a classified subset of messages of the classified subsets of messages is classified based at least partly on a user interface portion from which individual messages in the classified subset of messages were generated;
appending the classified subset of messages to a log file, the log file including a number of independently-accessible partitions;
generating a plurality of data streams using the classified subsets of messages, wherein a data stream of the plurality of data streams comprises the classified subset of messages;
determining, based at least partly on the message data, a subset of rental properties to be presented on a computing device;
determining a stream characteristic of the data stream;
detecting an anomaly associated with the data stream based at least partly on the stream characteristic; and
generating anomaly resolution data regarding the anomaly.
19. The non-transitory computer-readable storage of claim 18 , wherein determining the stream characteristic is performed in substantially real-time as data in the data stream transits between a broker computing device storing the classified subsets of messages and a consumer computing device configured to consume the data in the data stream.
20. The non-transitory computer-readable storage of claim 18 , wherein the process further comprises:
identifying a plurality of parametric values using (1) the stream characteristic and (2) a plurality of other stream characteristics;
identifying a correlated subset of the plurality of parametric values based at least partly on the stream characteristic and the plurality of other stream characteristics;
classifying the correlated subset of the plurality of parametric values as non-anomalous; and
generating an anomaly threshold based at least partly on the correlated subset of the plurality of parametric values.