Systems and methods for indexing and aggregating data records
Computer-implemented systems and methods are disclosed for indexing and aggregating proxy server data. In accordance with some embodiments, the systems comprise a memory device configured to store a set of instructions, and one or more processors configured to execute the set of instructions to receive a plurality of log lines of proxy server data comprising a plurality of data components and index one or more of the plurality of data components for the plurality of log lines. The processors also generate a plurality of log data records, each including the one or more indexed data components, and generate a summarized data record by aggregating one or more log data records having at least one matching indexed data component.
1. A system comprising:
one or more memory devices configured to store program instructions; and
one or more processors configured to execute the program instructions to:
access a plurality of data records comprising a plurality of data values;
generate an interactive user interface including:
a listing of the plurality of data records, and
one or more selectable columns associated with the data values of the plurality of data records;
receive user input selecting at least a first selectable column of the one or more selectable columns;
identify those of the plurality of data records having matching data values within the first selectable column;
determine measures of the identified data records, wherein the measures of the identified data records are based on aggregations of the identified data records;
update the interactive user interface to include a summarized data record table indicating the matching data values and the measures of the identified data records; and
generate, for storage, a summarized data record including the aggregations of the identified data records.
2. The system of claim 1 , wherein the updated interactive user interface further includes an indication of a space reduction achieved by the summarized data record table.
3. The system of claim 1 , wherein the summarized data record table does not include at least some non-matching data values of the identified data records.
4. The system of claim 1 , wherein the summarized data record table includes rows associated with each of the matching data values.
5. The system of claim 1 , wherein:
the summarized data record table does not include at least some non-matching data values of the identified data records, and
the summarized data record table includes rows associated with each of the matching data values.
6. The system of claim 5 , wherein the updated interactive user interface further includes an indication of a space reduction achieved by the summarized data record table.
7. The system of claim 1 , wherein the data values include one or more of a domain, an HTTP status, a protocol, an HTTP request type, a number of bytes received, a number of bytes sent, or a request date.
8. The system of claim 1 , wherein the measures of the identified data records corresponds to a number of aggregated data records having the matching data values.
9. A computer-implemented method comprising:
accessing a plurality of data records comprising a plurality of data values;
generating an interactive user interface including:
a listing of the plurality of data records, and
one or more selectable columns associated with the data values of the plurality of data records;
receiving user input selecting at least a first selectable column of the one or more selectable columns;
identifying those of the plurality of data records having matching data values within the first selectable column;
determining measures of the identified data records, wherein the measures of the identified data records are based on aggregations of the identified data records;
updating the interactive user interface to include a summarized data record table indicating the matching data values and the measures of the identified data records; and
generating, for storage, a summarized data record including the aggregations of the identified data records.
10. The computer-implemented method of claim 9 , wherein the updated interactive user interface further includes an indication of a space reduction achieved by the summarized data record table.
11. The computer-implemented method of claim 9 , wherein the summarized data record table does not include at least some non-matching data values of the identified data records.
12. The computer-implemented method of claim 9 , wherein the summarized data record table includes rows associated with each of the matching data values.
13. The computer-implemented method of claim 9 , wherein:
the summarized data record table does not include at least some non-matching data values of the identified data records, and
the summarized data record table includes rows associated with each of the matching data values.
14. The computer-implemented method of claim 13 , wherein the updated interactive user interface further includes an indication of a space reduction achieved by the summarized data record table.
15. A non-transitory computer-readable medium storing a set of instructions that are executable by one or more processors to cause a computing system to:
access a plurality of data records comprising a plurality of data values;
generate an interactive user interface including:
a listing of the plurality of data records, and
one or more selectable columns associated with the data values of the plurality of data records;
receive user input selecting at least a first selectable column of the one or more selectable columns;
identify those of the plurality of data records having matching data values within the first selectable column;
determine measures of the identified data records, wherein the measures of the identified data records are based on aggregations of the identified data records;
update the interactive user interface to include a summarized data record table indicating the matching data values and the measures of the identified data records; and
generate, for storage, a summarized data record including the aggregations of the identified data records.
16. The non-transitory computer-readable medium of claim 15 , wherein the updated interactive user interface further includes an indication of a space reduction achieved by the summarized data record table.
17. The non-transitory computer-readable medium of claim 15 , wherein the summarized data record table does not include at least some non-matching data values of the identified data records.
18. The non-transitory computer-readable medium of claim 15 , wherein the summarized data record table includes rows associated with each of the matching data values.
19. The non-transitory computer-readable medium of claim 15 , wherein:
the summarized data record table includes rows associated with each of the matching data values,
the summarized data record table does not include at least some non-matching data values of the identified data records, and
the summarized data record table includes rows associated with each of the matching data values.