Mechanism for reducing logging entries based on content
Embodiments relate to reducing logging entries based on contents are disclosed. According to the embodiments, the logging entries are converted into an image with pixels whose values are related with the log content and the image size is reduced by applying an image processing technology. The reduced image is converted back to the log entries. In this way, the number of logging entries is reduced without losing important information.
1. A device comprising:
at least one processor; and
at least one memory including computer program codes;
the at least one memory and the computer program codes are configured to, with the at least one processor, cause the device to:
for each logging entry of a set of logging entries, determine a pixel value for the logging entry based on a content of the logging entry;
generate a first image for the set of logging entries based on the pixel values;
generate a second image by reducing a size of the first image using a predetermined algorithm, and
generate a further set of logging entries by restoring the second image.
2. The device of claim 1 , wherein the device is caused to determine the pixel value for the logging entry by:
generating a signature for the logging entry, the signature being specific to the content;
performing normalization on the signature; and
determining a normalized value of the signature as the pixel value.
3. The device of claim 1 , wherein the device is caused to determine the pixel value for the logging entry by:
determining the pixel value for the logging entry based on a frequency of an occurrence of the logging entry in the set of logging entries.
4. The device of claim 3 , wherein the device is caused to determine the pixel value for the logging entry by:
if the logging entry occurs a predetermined number of times in the set of the logging entries, determining the pixel value to be a first value;
if the logging entry occurs more than the predetermined number of times in the set of the logging entries, determining the pixel value to be a second value, the second value being smaller than the first value.
5. The device of claim 1 , wherein the device is caused to generate the first image by:
sorting the pixel values in an order of arrival times of the respective logging entries; and
combining the sorted pixel values to generate the first image.
6. The device of claim 1 , wherein the device is caused to generate the second image by:
determining a set of pixels in the image with pixel values below a threshold value; and
generating the second image by removing the set of pixels from the first image.
7. A method comprising:
for each logging entry of a set of logging entries, determining a pixel value for the logging entry based on a content of the logging entry;
generating a first image for the set of logging entries based on the pixel values;
generating a second image by reducing a size of the first image using a predetermined algorithm; and
generating a further set of logging entries by restoring the second image.
8. The method of claim 7 , wherein determining the pixel value for the logging entry comprises:
generating a signature for the logging entry, the signature being specific to the content;
performing normalization on the signature; and
determining a normalized value of the signature as the pixel value.
9. The method of claim 7 , wherein determining the pixel value for the logging entry comprises:
determining the pixel value for the logging entry based on a frequency of an occurrence of the logging entry in the set of logging entries.
10. The method of claim 9 , wherein determining the pixel value for the logging entry comprises:
if the logging entry occurs a predetermined number of times in the set of the logging entries, determining the pixel value to be a first value;
if the logging entry occurs more than the predetermined number of times in the set of the logging entries, determining the pixel value to be a second value, the second value being smaller than the first value.
11. The method of claim 7 , wherein generating the first image comprises:
sorting the pixel values in an order of arrival times of the respective logging entries; and
combining the sorted pixel values to generate the first image.
12. The method of claim 7 , wherein generating the second image comprises:
determining a set of pixels in the image with pixel values below a threshold value; and
generating the second image by removing the set of pixels from the first image.
13. A non-transitory computer readable medium storing instructions thereon, the instructions, when executed by at least one processing unit of an apparatus, causing the apparatus to:
for each logging entry of a set of logging entries, determine a pixel value for the logging entry based on a content of the logging entry;
generate a first image for the set of logging entries based on the pixel values;
generate a second image by reducing a size of the first image using a predetermined algorithm; and
generating a further set of logging entries by restoring the second image.
14. The non-transitory computer readable medium storing instructions of claim 13 , wherein the apparatus is further caused to determine the pixel value for the logging entry by:
generating a signature for the logging entry, the signature being specific to the content;
performing normalization on the signature; and
determining a normalized value of the signature as the pixel value.
15. The non-transitory computer readable medium storing instructions of claim 13 , wherein the apparatus is further caused to determine the pixel value for the logging entry by:
determining the pixel value for the logging entry based on a frequency of an occurrence of the logging entry in the set of logging entries.
16. The non-transitory computer readable medium storing instructions of claim 15 , wherein the apparatus is further caused to determine the pixel value for the logging entry by:
if the logging entry occurs a predetermined number of times in the set of the logging entries, determining the pixel value to be a first value;
if the logging entry occurs more than the predetermined number of times in the set of the logging entries, determining the pixel value to be a second value, the second value being smaller than the first value.
17. The non-transitory computer readable medium storing instructions of claim 13 , wherein the apparatus is further caused to generate the first image by:
sorting the pixel values in an order of arrival times of the respective logging entries; and
combining the sorted pixel values to generate the first image.
18. The non-transitory computer readable medium storing instructions of claim 13 , wherein the apparatus is further caused to generate the second image by:
determining a set of pixels in the image with pixel values below a threshold value; and
generating the second image by removing the set of pixels from the first image.