Compression-based detection of inefficiency in external services
View Patent ↗The disclosed embodiments provide a system for detecting and managing inefficiency in external services. During operation, the system obtains a snapshot of a data stream transmitted over an external service from a computer system at a first time. Next, the system applies a compression technique to the snapshot to obtain a set of inefficiency metrics for a set of data elements in the snapshot. The system then outputs the set of inefficiency metrics with additional attributes of the data stream to improve identification of inefficiency in the data stream.
1. A method, comprising:
obtaining a snapshot of a data stream transmitted over an external service from a computer system at a first time;
applying, by a processor, a compression technique to the snapshot to obtain a set of inefficiency metrics for a first set of data elements in the snapshot;
grouping, by a first attribute, a first subset of the first set of data elements;
grouping, by a second attribute, a second subset of the first set of data elements;
applying the compression technique to:
the first subset to obtain a first group inefficiency metric for the first subset; and
the second subset to obtain a second group inefficiency metric for the second subset;
using the first and second group inefficiency metrics to identify contributions of the first and second subsets to an inefficiency in the data stream; and
outputting one or more inefficiency metrics to improve management of the inefficiency in the data stream, wherein one or more of the first group inefficiency metric and the second group inefficiency metric are included in the outputted inefficiency metrics.
2. The method of claim 1 , wherein the first attribute comprises at least one of:
a port;
a service type;
a protocol;
a network stack layer;
a data element type;
an address;
a source; and
a destination.
3. The method of claim 1 , wherein the inefficiency is
a schema inefficiency.
4. The method of claim 1 , wherein:
the external service comprises a networking service, and
the first set of data elements comprises a network packet.
5. The method of claim 1 , wherein the external service comprises an interconnect service.
6. The method of claim 1 , wherein outputting the set of inefficiency metrics with the additional attributes of the first set of data elements comprises at least one of:
displaying a ranking of the first set of data elements by the first set of inefficiency metrics; and
identifying a subset of the first set of data elements as candidates for improving the inefficiency.
7. The method of claim 1 , wherein outputting the set of inefficiency metrics with the additional attributes of the data stream comprises:
outputting, based on the first set of inefficiency metrics, a security flaw in the data stream.
8. The method of claim 1 , wherein the set of inefficiency metrics comprises at least one of:
a redundancy; and
a compression ratio.
9. The method of claim 1 , wherein the inefficiency is a request strategy inefficiency.
10. The method of claim 1 , wherein the second subset is a subset of the first subset.
11. An apparatus, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a snapshot of a data stream transmitted over an external service from a computer system at a first time;
apply a compression technique to the snapshot to obtain a set of inefficiency metrics for a first set of data elements in the snapshot;
group, by a first attribute, a first subset of the set of data elements;
group, by a second attribute, a second subset of the first set of data elements;
apply the compression technique to:
the first subset to obtain a first group inefficiency metric for the first subset; and
the second subset to obtain a second group inefficiency metric for the second subset;
use the first and second group inefficiency metrics to identify contributions of the first and second subsets to an inefficiency in the data stream; and
output one or more inefficiency metrics to improve management of the inefficiency in the data stream, wherein one or more of the first group inefficiency metric and the second group inefficiency metric are included in the outputted inefficiency metrics.
12. The apparatus of claim 11 , wherein the first attribute comprises at least one of:
a port;
a service type;
a protocol;
a network stack layer;
a data element type;
an address;
a source; and
a destination.
13. The apparatus of claim 11 , wherein the inefficiency is
a schema inefficiency.
14. The apparatus of claim 11 , wherein:
the external service comprises a networking service, and
the first set of data elements comprises a network packet.
15. The apparatus of claim 11 , wherein the external service comprises an interconnect service.
16. The apparatus of claim 11 , wherein outputting the set of inefficiency metrics with the additional attributes of the data stream comprises:
outputting, based on the first set of inefficiency metrics, a security flaw in the data stream.
17. The apparatus of claim 11 , wherein the inefficiency isa request strategy inefficiency.
18. The apparatus of claim 11 , wherein the second subset is a subset of the first subset.
19. A system, comprising:
an analysis module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to:
obtain a snapshot of a data stream transmitted over an external service from a computer system at a first time;
apply a compression technique to the snapshot to obtain a set of inefficiency metrics for a first set of data elements in the snapshot;
group, by a first attribute, a first subset of the first set of data elements;
group, by a second attribute, a second subset of the first set of data elements;
apply the compression technique to:
the first subset to obtain a first group inefficiency metric for the first subset; and
the second subset to obtain a second group inefficiency metric for the second subset; and
use the first and second group inefficiency metrics to identify contributions of the first and second subsets to an inefficiency in the data stream; and
a management module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to output one or more inefficiency metrics to improve management of the identification of inefficiency in the data stream, wherein one or more of the first group inefficiency metric and the second group inefficiency metric are included in the outputted inefficiency metrics.
20. The system of claim 19 , wherein the second subset is a subset of the first subset.