Space-efficient storage of system-monitoring data
An amount of storage space required to maintain counter information for a storage system is reduced without reducing a temporal granularity or tracking granularity of the counter information. Rather than periodically recording actual (i.e., raw) counter values for counters, difference (i.e., delta) values may be recorded. For a given counter, a difference (delta value) between a value of the counter for a given point in time (PIT) and a value of the counter for a previous PIT may be determined, and this delta value may be stored as opposed to storing the raw counter value. This delta value may be a significantly smaller value than the raw value. To further reduce the amount of storage space required, no value may be stored for a counter for a given PIT if it is determined that there is no difference between a counter value for the given PIT and a previous PIT.
1. A method comprising:
receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
determining whether there is a difference between the first counter value and the second counter value;
if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
refraining from persisting the first counter value;
storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
after the first point in time, determining that there was not a difference between the first counter value and the second counter value based on the metadata.
2. The method of claim 1 , further comprising:
if it is determined that there is not any difference between the first counter value and the second counter value, refraining from storing any value for the first counter for the first point in time.
3. The method of claim 1 , further comprising:
after the first point in time, generating the first counter value based on the determination that there was not a difference between the first counter value and the second counter value.
4. The method of claim 1 , wherein the first counter counts occurrences of execution of a specific line of code on the storage system.
5. A method comprising:
receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
determining whether there is a difference between the first counter value and the second counter value;
if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
refraining from persisting the first counter value;
storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
at a point in time after the first point in time, generating the first counter value from the metadata and the information associating the first counter with the difference for the first point in time.
6. The method of claim 5 , wherein the metadata is stored in at least a first relational database table,
wherein difference information determined from the counter information, including the information associating the first counter with the difference for the first point in time, is stored in at least a second relational database table, and
wherein generating the first counter value includes executing a structured query language (SQL) join operation between the at least first relational database table and the at least second relational database table.
7. A storage system comprising;
one or more processors: and
a memory comprising executable code that, when executed, performs a method including:
receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
determining whether there is a difference between the first counter value and the second counter value;
if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
refraining from persisting the first counter value;
storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
after the first point in time, determining that there was not a difference between the first counter value and the second counter value based on the metadata.
8. The system of claim 7 , wherein the method further comprises:
if it is determined that there is not any difference between the first counter value and the second counter value, refraining from storing any value for the first counter for the first point in time.
9. The system of claim 7 , wherein the method further comprises:
after the first point in time, generating the first counter value based on the determination that there was not a difference between the first counter value and the second counter value.
10. A storage system comprising:
one or more processors: and
a memory comprising executable code that, when executed, performs a method including:
receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
determining whether there is a difference between the first counter value and the second counter value;
if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
refraining from persisting the first counter value;
storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
at a point in time after the first point in time, generating the first counter value from the metadata and the information associating the first counter with the difference for the first point in time.
11. The system of claim 10 , wherein the metadata is stored in at least a first relational database table,
wherein difference information determined from the counter information, including the information associating the first counter with the difference for the first point in time, is stored in at least a second relational database table, and
wherein generating the first counter value includes executing a structured query language (SQL) join operation between the at least first relational database table and the at least second relational database table.
12. The system of claim 11 , wherein the first counter counts occurrences of execution of a specific line of code on the storage system.
13. One or more non-transitory computer-readable media having software stored thereon, the software comprising:
executable code that controls receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
executable code that controls obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
executable code that controls determining whether there is a difference between the first counter value and the second counter value;
executable code that controls, if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
executable code that controls refraining from persisting the first counter value;
executable code that controls storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
executable code that controls, after the first point in time, determining that there was not a difference between the first counter value and the second counter value based on the metadata.
14. The one or more non-transitory computer-readable media of claim 13 , wherein the software further comprises:
executable code that controls, if it is determined that there is not any difference between the first counter value and the second counter value, refraining from storing any value for the first counter for the first point in time.
15. The one or more non-transitory computer-readable media of claim 13 , wherein the software further comprises:
executable code that controls, after the first point in time, generating the first counter value based on the determination that there was not a difference between the first counter value and the second counter value.
16. One or more non-transitory computer-readable media having software stored thereon, the software comprising:
executable code that controls receiving counter information for a first point in time for a plurality of counters on a storage system, the counter information including a first counter value specifying a value of a first counter at the first point in time;
executable code that controls obtaining a second counter value specifying a value of the first counter at a second point in time before the first point in time;
executable code that controls determining whether there is a difference between the first counter value and the second counter value;
executable code that controls, if it is determined that there is a difference, persisting information associating the first counter with the difference for the first point in time;
executable code that controls refraining from persisting the first counter value;
executable code that controls storing metadata reflecting that the counter information, including first counter information for the first counter, was generated for the first point in time; and
executable code that controls, at a point in time after the first point in time, generating the first counter value from the metadata and the information associating the first counter with the difference for the first point in time.
17. The one or more non-transitory computer-readable media of claim 16 , wherein the metadata is stored in at least a first relational database table,
wherein difference information determined from the counter information, including the information associating the first counter with the difference for the first point in time, is stored in at least a second relational database table, and
wherein generating the first counter value includes executing a structured query language (SQL) join operation between the at least first relational database table and the at least second relational database table.