Tree-based learning of application programming interface specification
A cybersecurity appliance monitoring application traffic to a web application programming interface (API) dynamically updates tree structures for the web API using the application traffic. An API tree generator generates batches of API trees from paths indicated in the application traffic. An API tree merger/pruner updates the generated batches of API trees with various merging, pruning, compacting, and malicious detection operations on the generated batches of API trees. The cybersecurity appliance implements the updated API trees with an API agent that filters the application traffic prior to processing by the web API.
1 . A method comprising:
dynamically updating application programming interface (API) trees while filtering application traffic with the API trees, wherein dynamically updating the API trees while filtering application traffic with the API trees comprises,
based on detecting a first API request in application traffic, extracting a first path from the first API request, wherein the first path corresponds to one or more resources corresponding to an API;
determining whether the first path from the first API request corresponds to a second path in a first tree for the API, wherein the first tree comprises at least one of common paths and common nodes in a first plurality of API requests of the application traffic;
based on determining that the first path from the first API request does not correspond to the second path in the first tree for the API, filtering the first API request from the application traffic; and
updating the first tree for the API based on a second plurality of API requests detected in the application traffic including the first API request, wherein updating the first tree for the API comprises removing at least one of malicious nodes and malicious paths from the first tree.
2 . The method of claim 1 , further comprising, based on determining that the first path from the first API request corresponds to the second path in the first tree, communicating a query indicated in the first API request to the one or more resources managed by the API.
3 . The method of claim 1 , wherein the first plurality of API requests of the application traffic comprise API requests filtered according to one or more security policies.
4 . The method of claim 1 , wherein determining that the first path from the first API request corresponds to the second path in the first tree for the API comprises, at each node in a traversal of the first tree, verifying whether the node matches a corresponding node in the first path.
5 . The method of claim 1 , wherein the at least one of common paths and common nodes in the first plurality of API requests comprise paths and nodes in a plurality of trees that were merged into the first tree.
6 . The method of claim 5 , wherein updating the first tree for the API comprises:
detecting the at least one of malicious nodes and malicious paths; and
pruning the at least one of malicious nodes and malicious paths prior to merging the plurality of trees.
7 . The method of claim 1 , wherein updating the first tree for the API comprises:
determining that query results from communicating the second plurality of API requests in the application traffic to corresponding resources are responsive;
storing the second plurality of API requests in local memory; and
updating the first tree based on paths in the second plurality of API requests.
8 . A non-transitory machine-readable medium having program code stored thereon that is executable by a computing device, the program code comprising instructions to:
dynamically update application programming interface (API) trees while filtering application traffic with the API trees, wherein the instructions to dynamically update the API trees while filtering traffic with the API trees comprise instructions to,
based on detecting a first API request in application traffic, extract a first path from the first API request, wherein the first path corresponds to one or more resources managed by an API;
traverse a first tree for the API to determine whether the first path from the first API request matches a second path of the first tree, wherein the first tree comprises at least one of common paths and common nodes in a first plurality of API requests of the application traffic;
based on determining that the first path from the first API request does not match the second path in the first tree for the API, filter the first API request from the application traffic; and
update the first tree for the API based on a second plurality of API requests detected in the application traffic including the first API request, wherein updating the first tree for the API comprises removing at least one of malicious nodes and malicious paths from the first tree.
9 . The non-transitory machine-readable medium of claim 8 , wherein the program code further comprises instructions to, based on determining that the first path from the first API request matches the second path in the first tree, communicate a query indicated in the first API request to the one or more resources managed by the API.
10 . The non-transitory machine-readable medium of claim 8 , wherein the first plurality of API requests of the application traffic comprise API requests filtered according to one or more security policies.
11 . The non-transitory machine-readable medium of claim 8 , wherein the at least one of common paths and common nodes in the first plurality of API requests comprise paths and nodes in a plurality of trees that were merged into the first tree.
12 . The non-transitory machine-readable medium of claim 11 , wherein the instructions to update the first tree for the API comprise instructions to:
detect the at least one of malicious nodes and malicious paths; and
prune the at least one of malicious nodes and malicious paths prior to merging the plurality of trees.
13 . The non-transitory machine-readable medium of claim 8 , wherein the instructions to update the first tree for the API comprise instructions to:
determine that query results from communicating the second plurality of API requests in the application traffic to corresponding resources are responsive;
store the second plurality of API requests in local memory; and
update the first tree based on paths in the second plurality of API requests.
14 . The non-transitory machine-readable medium of claim 9 , wherein the instructions to traverse the first tree for the API to determine whether the first path from the first API request matches the second path of the first tree comprise instructions to, at each node or representative node in a traversal of the first tree, determine whether the node or representative node matches a corresponding node in the first path.
15 . An apparatus comprising:
a processor; and
a non-transitory machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to:
dynamically update application programming interface (API) trees while filtering application traffic with the API trees, wherein the instructions to dynamically update the API trees while filtering traffic with the API trees comprise instructions executable by the processor to cause the apparatus to,
based on detecting a first API request in application traffic, extract a first path from a first API request, wherein the first path corresponds to one or more resources managed by an API;
traverse a first tree for the API to determine whether the first path from the first API request matches a second path of the first tree, wherein the first tree comprises at least one of compacted trees and merged trees, wherein the at least one of compacted trees and merged trees comprise at least one of common paths and common nodes in a first plurality of API requests of the application traffic;
based on determining that the first path from the first API request does not match the second path in the first tree for the API, filter the first API request from the application traffic; and
update the first tree for the API based on a second plurality of API requests detected in the application traffic including the first API request, wherein updating the first tree for the API comprises removing at least one of malicious nodes and malicious paths from the first tree.
16 . The apparatus of claim 15 , wherein the machine-readable medium further has stored thereon instructions executable by the processor to cause the apparatus to, based on determining that the first path from the first API request matches the second path in the first tree, communicate a query indicated in the first API request to the one or more resources managed by the API.
17 . The apparatus of claim 15 , wherein the first plurality of API requests of the application traffic comprise API requests filtered according to one or more security policies.
18 . The apparatus of claim 15 , wherein the instructions to update the first tree for the API comprise instructions executable by the processor to cause the apparatus to:
detect the at least one of malicious nodes and malicious paths; and
prune the at least one of malicious nodes and malicious paths from the at least one of compacted trees and merged trees.
19 . The apparatus of claim 15 , wherein the instructions to update the first tree for the API comprise instructions executable by the processor to cause the apparatus to:
determine that query results from communicating the second plurality of API requests in the application traffic to corresponding resources are responsive;
store the second plurality of API requests in local memory; and
update the first tree based on paths in the second plurality of API requests.
20 . The apparatus of claim 15 , wherein the instructions to traverse the first tree for the API to determine whether the first path from the first API request matches the second path of the first tree comprise instructions executable by the processor to cause the apparatus to, at each node or representative node in a traversal of the first tree, determine whether the node or representative node matches a corresponding node in the first path.