Advanced intelligence engine
An advanced intelligence engine (AIE) for use in identifying what may be complex events or developments on one or more data platforms or networks from various types of structured or normalized data generated by one or more disparate data sources. The AIE may conduct one or more types of quantitative, correlative, behavioral and corroborative analyses to detect events from what may otherwise be considered unimportant or non-relevant information spanning one or more time periods. Events generated by the AIE may be passed to an event manager to determine whether further action is required such as reporting, remediation, and the like.
1. A method for use in monitoring one or more platforms of one or more data systems, comprising:
first evaluating, by a processor using a first rule block, structured data received from one or more platforms over at least one communications network;
first determining, from the first evaluating, that a result is one of at least first and second outcomes;
accessing, by the processor, a linking relationship object in the first rule block to identify a data field in the structured data;
extracting, by the processor, a content of the data field from the structured data;
second evaluating, by the processor using a second rule block, structured data associated with the extracted content received from the one or more platforms;
second determining, from the second evaluating, whether a result is one of at least first and second outcomes; and
analyzing the results of the first and second determining to determine an event of interest.
2. The method of claim 1 , further comprising:
passing the extracted content to the second rule block, wherein the second evaluating is performed in response to receipt of the extracted content by the second rule block.
3. The method of claim 1 , wherein the second evaluating comprises:
searching a plurality of entries of an indexing structure of the second rule block for an entry that corresponds to the extracted content.
4. The method of claim 3 , wherein the second evaluating comprises:
filtering, by the processor using the second rule block, the structured data to generate second rule block filtered data, wherein each entry of the plurality of entries of the indexing structure of the second rule block indicates a presence of a corresponding portion of the second rule block filtered data.
5. The method of claim 4 , wherein the filtering occurs before the first determining.
6. The method of claim 4 , further including:
determining that an entry exists in the plurality of entries of the indexing structure of the second rule block that corresponds to the extracted content;
obtaining the portion of the second rule block filtered data that corresponds to the determined entry; and
generating a satisfied condition object based on the obtained portion of the second rule block filtered data, wherein the analyzing includes analyzing the satisfied condition object of the second rule block.
7. The method of claim 4 , further comprising:
accessing, by the processor, a linking relationship object in the second rule block to identify a data field; and
generating the indexing structure of the second rule block based on the identified data field from the linking relationship object of the second rule block.
8. The method of claim 7 , wherein the second rule block filtered data includes a plurality of portions of structured data, and further comprising:
obtaining, by the processor, a content of the identified data field for each of the plurality of structured data portions, wherein the generating includes using the content of the identified data field from the linking relationship object of the second rule block from each of the plurality of structured data portions of the second rule block filtered data to generate the indexing structure of the second rule block.
9. The method of claim 7 , wherein the first evaluating comprises:
filtering, by the processor using the first rule block, the at least some of the data to generate first rule block filtered data; and
generating an indexing structure of the first rule block based on the identified data field from the linking relationship object of the first rule block, wherein the indexing structure of the first rule block includes a plurality of entries, and wherein each entry of the plurality of entries of the indexing structure of the first rule block indicates a presence of a corresponding portion of the first rule block filtered data.
10. The method of claim 9 , wherein the identified data field from the linking relationship object of the first rule block is different than the identified data field from the linking relationship object of the second rule block.
11. The method of claim 9 , wherein the first rule block filtered data includes a plurality of portions of structured data, and further comprising:
obtaining, by the processor, the content of the identified data field for each of the plurality of structured data portions, wherein the generating includes using the content of the identified data field from the linking relationship object of the first rule block from each of the plurality of structured data portions of the first rule block filtered data to create the indexing structure of the first rule block.
12. The method of claim 3 , wherein the result of the second determining corresponds to data associated with the extracted content and evaluated by the second rule block.
13. A non-transitory, computer-readable storage medium, storing program instructions that when executed on one or more computers cause the one or more computers to perform:
first evaluating, using a first rule block, structured data received from one or more platforms over at least one communications network;
first determining, from the first evaluating, that a result is one of at least first and second outcomes;
accessing a linking relationship object in the first rule block to identify a data field in the structured data;
extracting a content of the data field from the structured data;
second evaluating, using a second rule block, structured data associated with the extracted content received from the one or more platforms;
second determining, from the second evaluating, whether a result is one of at least first and second outcomes; and
analyzing the results of the first and second determining to determine an event of interest.
14. The non-transitory, computer-readable storage medium of claim 13 , further storing program instructions that when executed on one or more computers cause the one or more computers to perform:
passing the extracted content to the second rule block, wherein the second evaluating is performed in response to receipt of the extracted content by the second rule block.
15. The non-transitory, computer-readable storage medium of claim 13 , wherein the second evaluating comprises:
searching a plurality of entries of an indexing structure of the second rule block for an entry that corresponds to the extracted content.
16. The non-transitory, computer-readable storage medium of claim 15 , wherein the second evaluating comprises:
filtering, using the second rule block, the structured data to generate second rule block filtered data, wherein each entry of the plurality of entries of the indexing structure of the second rule block indicates a presence of a corresponding portion of the second rule block filtered data.
17. The non-transitory, computer-readable storage medium of claim 16 , wherein the filtering occurs before the first determining.
18. The non-transitory, computer-readable storage medium of claim 16 , further storing program instructions that when executed on one or more computers cause the one or more computers to perform:
determining that an entry exists in the plurality of entries of the indexing structure of the second rule block that corresponds to the extracted content;
obtaining the portion of the second rule block filtered data that corresponds to the determined entry; and
generating a satisfied condition object based on the obtained portion of the second rule block filtered data, wherein the analyzing includes analyzing the satisfied condition object of the second rule block.
19. The non-transitory, computer-readable storage medium of claim 16 , further comprising:
accessing a linking relationship object in the second rule block to identify a data field; and
generating the indexing structure of the second rule block based on the identified data field from the linking relationship object of the second rule block.
20. The non-transitory, computer-readable storage medium of claim 19 , wherein the second rule block filtered data includes a plurality of portions of structured data, and further comprising:
obtaining a content of the identified data field for each of the plurality of structured data portions, wherein the generating includes using the content of the identified data field from the linking relationship object of the second rule block from each of the plurality of structured data portions of the second rule block filtered data to generate the indexing structure of the second rule block.
21. The non-transitory, computer-readable storage medium of claim 19 , wherein the first evaluating comprises:
filtering, using the first rule block, the at least some of the data to generate first rule block filtered data;
generating an indexing structure of the first rule block based on the identified data field from the linking relationship object of the first rule block, wherein the indexing structure of the first rule block includes a plurality of entries, and wherein each entry of the plurality of entries of the indexing structure of the first rule block indicates a presence of a corresponding portion of the first rule block filtered data.
22. The non-transitory, computer-readable storage medium of claim 21 , wherein the identified data field from the linking relationship object of the first rule block is different than the identified data field from the linking relationship object of the second rule block.
23. The non-transitory, computer-readable storage medium of claim 21 , wherein the first rule block filtered data includes a plurality of portions of structured data, and further storing program instructions that when executed on one or more computers cause the one or more computers to perform:
obtaining the content of the identified data field for each of the plurality of structured data portions, wherein the generating includes using the content of the identified data field from the linking relationship object of the first rule block from each of the plurality of structured data portions of the first rule block filtered data to create the indexing structure of the first rule block.
24. The non-transitory, computer-readable storage medium of claim 15 , wherein the result of the second determining corresponds to data associated with the extracted content and evaluated by the second rule block.