System and method for automatically recommending logs for low-cost tier storage
A system and method for managing logs from computing environments uses a rate change in a rate of occurrence of same event type logs from a base time window to a current time window for each of the event types to identify candidate event types for a particular tier log storage. The rate changes of the event types are checked against a threshold rate change range to identify the candidate event types. In response to selection of some of the candidate event types, the logs in the selected candidate event types are transferred to the particular tier log storage.
1. A computer-implemented method for managing logs from computing environments, the method comprising:
receiving logs from a computing environment;
grouping the logs into event types using a log classification algorithm;
computing a rate change in a rate of occurrence of same event type logs from a base time window to a current time window for each of the event types;
checking the rate changes of the event types against a threshold rate change range to identify candidate event types for a particular tier log storage;
sending the candidate event types for selection for the particular tier log storage; and
in response to selection of some of the candidate event types, transferring the logs in the candidate event types that are selected to the particular tier log storage.
2. The computer-implemented method of claim 1 , further comprising modifying the threshold rate change range based on selection of the candidate event types.
3. The computer-implemented method of claim 2 , wherein modifying the threshold rate change range includes changing a threshold value of the threshold rate change range when an acceptance rate of the candidate event types at the threshold value is less than an acceptance threshold.
4. The computer-implemented method of claim 3 , wherein changing the threshold value of the threshold rate change range includes changing the threshold value of the threshold rate change range such that the threshold rate change range is shortened.
5. The computer-implemented method of claim 2 , wherein modifying the threshold rate change range includes changing a threshold value of the threshold rate change range when an acceptance rate of the candidate event types at the threshold value is greater than an acceptance threshold.
6. The computer-implemented method of claim 5 , wherein changing the threshold value of the threshold rate change range includes changing the threshold value of the threshold rate change range such that the threshold rate change range is lengthened.
7. The computer-implemented method of claim 1 , wherein the threshold rate change range is a local threshold rate change range for the computing environment that is distinct from a global threshold rate change range, the global threshold rate change range being used as a default range for new computing environments for a log management service.
8. The computer-implemented method of claim 1 , wherein the duration of the base time window equals the duration of the current time window and wherein the base time window precedes the current time window.
9. The computer-implemented method of claim 1 , wherein sending the candidate event types includes sending a representative log from each of the candidate event types to a user interface for user selection.
10. A non-transitory computer-readable storage medium containing program instructions for managing logs from computing environments, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform steps comprising:
receiving logs from a computing environment;
grouping the logs into event types using a log classification algorithm;
computing a rate change in a rate of occurrence of same event type logs from a base time window to a current time window for each of the event types;
checking the rate changes of the event types against a threshold rate change range to identify candidate event types for a particular tier log storage;
sending the candidate event types for selection for the particular tier log storage; and
in response to selection of some of the candidate event types, transferring the logs in the candidate event types that are selected to the particular tier log storage.
11. The computer-readable storage medium of claim 10 , wherein the steps further comprise modifying the threshold rate change range based on selection of the candidate event types.
12. The computer-readable storage medium of claim 11 , wherein modifying the threshold rate change range includes changing a threshold value of the threshold rate change range when an acceptance rate of the candidate event types at the threshold value is less than an acceptance threshold.
13. The computer-readable storage medium of claim 12 , wherein changing the threshold value of the threshold rate change range includes changing the threshold value of the threshold rate change range such that the threshold rate change range is shortened.
14. The computer-readable storage medium of claim 11 , wherein modifying the threshold rate change range includes changing a threshold value of the threshold rate change range when an acceptance rate of the candidate event types at the threshold value is greater than an acceptance threshold.
15. The computer-readable storage medium of claim 14 , wherein changing the threshold value of the threshold rate change range includes changing the threshold value of the threshold rate change range such that the threshold rate change range is lengthened.
16. The computer-readable storage medium of claim 10 , wherein the threshold rate change range is a local threshold rate change range for the computing environment that is distinct from a global threshold rate change range, the global threshold rate change range being used as a default range for new computing environments for a log management service.
17. The computer-readable storage medium of claim 10 , wherein the duration of the base time window equals the duration of the current time window and wherein the base time window precedes the current time window.
18. The computer-readable storage medium of claim 10 , wherein sending the candidate event types includes sending a representative log from each of the candidate event types to a user interface for user selection.
19. A system comprising:
memory; and
at least one processor configured to:
receive logs from a computing environment;
group the logs into event types using a log classification algorithm;
compute a rate change in a rate of occurrence of same event type logs from a base time window to a current time window for each of the event types;
check the rate changes of the event types against a threshold rate change range to identify candidate event types for a particular tier log storage;
send the candidate event types for selection for the particular tier log storage; and
in response to selection of some of the candidate event types, transfer the logs in the candidate event types that are selected to the particular tier log storage.
20. The system of claim 19 , wherein the at least one processor is further configured to modify the threshold rate change range based on selection of the candidate event types.