Data lake-enabled security platform
Providing a data lake-enabled security platform, including: storing security data associated with a customer in a data lake comprising a plurality of storage environments implemented in different cloud environments of a plurality of cloud environments; generating, based on a plurality of records in the security data, an abstracted security record describing one or more derived insights of the security data; and providing access to the abstracted security record to one or more users associated with the customer.
1. A method of providing a data lake-enabled security platform, the method comprising:
storing security data associated with a customer in a data lake comprising a plurality of storage environments implemented in different cloud environments;
generating, based on a plurality of records in the security data, an abstracted security record describing one or more insights derived from the security data, wherein generating the abstracted security record comprises processing, by a processing engine, portions of the security data to generate abstracted security records, wherein the processing engine generates the abstracted security records based on one or more customer-selected modules for generating abstracted security records, wherein the one or more customer-selected modules control how the plurality of records are generated into the abstracted security record; and
providing access to the abstracted security record in the data lake to one or more users associated with the customer.
2. The method of claim 1 , wherein the security data comprises a plurality of raw security records stored in a lower storage tier relative to the abstracted security record.
3. The method of claim 1 , wherein the security data comprises a plurality of raw security records stored across the plurality of storage environments.
4. The method of claim 1 , wherein the security data comprises a plurality of processed security records stored across the plurality of storage environments.
5. The method of claim 1 , wherein the abstracted security record is included in a plurality of abstracted security records stored across the plurality of storage environments.
6. The method of claim 1 , further comprising storing data associating the abstracted security record with the plurality of records.
7. The method of claim 1 , wherein generating the abstracted security record comprises allocating processing of at least a portion of the security data across the plurality of cloud environments.
8. The method of claim 7 , wherein allocating processing of the portion of the security data is based on one or more processing costs for the plurality of records.
9. The method of claim 7 , wherein allocating processing of the portion of the security data is based on current resource utilizations for the plurality of cloud environments.
10. The method of claim 1 , wherein the data lake provides a unified view of data stored across the plurality of cloud environments.
11. The method of claim 1 , wherein the plurality of records correspond to a plurality of instances of detected activity associated with the customer and wherein the abstracted security record describes the detected activity.
12. The method of claim 1 , wherein the plurality of records is associated with a particular event and wherein the abstracted security record describes the particular event.
13. The method of claim 12 , wherein the particular event comprises a detected anomaly.
14. The method of claim 12 , wherein the particular event comprises a detected security threat.
15. The method of claim 1 , wherein the plurality of records is associated with a particular event and wherein the abstracted security record comprises one or more suggested actions associated with the particular event occurring.
16. The method of claim 1 , wherein access to the abstracted security record is provided via one or more of a query language or an application program interface (API).
17. The method of claim 16 further comprising generating a polygraph.
18. A computer program product for a data lake-enabled security platform, the computer program product disposed on a non-transitory computer readable medium, the computer program product including computer program instructions configurable to carry out the steps of:
storing security data associated with a customer in a data lake comprising a plurality of storage environments implemented in different cloud environments of a plurality of cloud environments;
generating, based on a plurality of records in the security data, an abstracted security record describing one or more derived insights of the security data, wherein generating the abstracted security record comprises processing, by a processing engine, portions of the security data to generate abstracted security records, wherein the processing engine generates the abstracted security records based on one or more customer-selected modules for generating abstracted security records, wherein the one or more customer-selected modules control how the plurality of records are generated into the abstracted security record; and
providing access to the abstracted security record in the data lake to one or more users associated with the customer.
19. The computer program product of claim 18 , wherein the security data comprises a plurality of raw security records stored in a lower storage tier relative to the abstracted security record.
20. The computer program product of claim 18 , wherein the security data comprises a plurality of raw security records stored across the plurality of storage environments.