Hybrid data scan pipeline reducing response latency and increasing attack scanning accuracy
Methods, systems, and computer program products provide a hybrid data scan pipeline or detector that reduces (as compared to conventional storage operations) response latency and increases scanning accuracy of encryption attacks such as ransomware attacks. For example, a frontend of a storage platform receiving an incoming data object may scan a portion of the data object for a change of an entropy level. The portion scanned may be insignificant relative to the overall size of the data object. As such, the operations of the frontend would place an insignificant delay to the overall storage processing. Other portions of the data object will be processed at a backend of the storage platform. For example, subsequent to receiving the data object, a change of entropy level of the other portions is scanned for detecting ransomware attacks.
1 . A method comprising:
receiving, at a frontend of a storage platform, an incoming object having a first segment and a second segment different from the first segment, the second segment including a number of parallel segments processed by two or more object storage daemons (OSDs);
scanning inline, by a processing device of the frontend of the storage platform, the first segment of the incoming object for a change of an entropy level of the first segment, wherein the scanning inline does not scan the second segment of the incoming object;
scanning offline, subsequent to the receiving and by a backend of the storage platform, the second segment of the incoming object for a change of an entropy level of the second segment, the backend being more computationally powerful than the frontend, wherein the scanning offline of the second segment comprises scanning, by the backend of the storage platform, the second segment of the incoming object during processing of the incoming object for storage at the backend of the storage platform; and
determining an encryption of the incoming object when the change of the entropy level of the first segment or the second segment exceeds a threshold value based on the scanning inline of the first segment and the scanning offline of the second segment.
2 . The method of claim 1 , wherein the change of the entropy level of the first segment and the change of the entropy level of the second segment indicates an encryption by a ransomware to the incoming object.
3 . The method of claim 1 , further comprising:
generating a warning upon detecting the change of the entropy level of the first segment exceeding the threshold value indicating a ransomware encryption; and
generating a metadata record of the scanning offline of the second segment, regardless whether the change of the entropy level of the second segment exceeds the threshold value.
4 . The method of claim 3 , further comprising:
responsive to detecting that the change of the entropy level of the second segment exceeds the threshold value, providing the generated metadata record to an external policy engine for verification of the ransomware encryption.
5 . The method of claim 1 , wherein the first segment comprises an initial subset of the incoming object.
6 . The method of claim 1 , wherein:
the storage platform implements object storage on a distributed computer cluster and provides integrated interfaces for at least one of: object-level, block-level, or file-level storage;
the frontend comprises a presentation layer or protocol of a reliable, autonomous, distributed object store (RADOS) gateway (RGW); and
the backend comprises a data access layer or storage of the two or more OSDs.
7 . A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processing device, cause the processing device to:
receive, at a frontend of a storage platform, an incoming object having a first segment and a second segment different from the first segment, the second segment including a number of parallel segments processed by two or more object storage daemons (OSDs);
scan inline, by the processing device of the frontend of the storage platform, the first segment of the incoming object for a change of an entropy level of the first segment, wherein the scanning inline does not scan the second segment of the incoming object;
scan offline, subsequent to the receiving and by a backend of the storage platform, the second segment of the incoming object for a change of an entropy level of the second segment, the backend being more computationally powerful than the frontend and configured to scan the second segment of the incoming object during processing of the incoming object for storage at the backend of the storage platform; and
determine an encryption of the incoming object when the change of the entropy level of the first segment or the second segment exceeds a threshold value based on the scanning inline of the first segment and the scanning offline of the second segment.
8 . The non-transitory computer-readable storage medium of claim 7 , wherein the change of the entropy level of the first segment and the change of the entropy level of the second segment indicates an encryption by a ransomware to the incoming object.
9 . The non-transitory computer-readable storage medium of claim 7 , further comprises instructions stored thereon to cause the processing device to:
generate a warning upon detecting the change of the entropy level of the first segment exceeding the threshold value indicating a ransomware encryption; and
generate a metadata record of the scanning offline of the second segment, regardless whether the change of the entropy level of the second segment exceeds the threshold value.
10 . The non-transitory computer-readable storage medium of claim 9 , further comprises instructions stored thereon to cause the processing device to:
responsive to detecting that the change of the entropy level of the second segment exceeds the threshold value, provide the generated metadata record to an external policy engine for verification of the ransomware encryption.
11 . The non-transitory computer-readable storage medium of claim 7 , wherein the first segment comprises an initial subset of the incoming object.
12 . The non-transitory computer-readable storage medium of claim 7 , wherein:
the storage platform implements object storage on a distributed computer cluster and provides integrated interfaces for at least one of: object-level, block-level, or file-level storage;
the frontend comprises a presentation layer or protocol of a reliable, autonomous, distributed object store (RADOS) gateway (RGW); and
the backend comprises a data access layer or storage of the two or more OSDs.
13 . A system comprising:
a memory; and
a processing device coupled to the memory, to:
receive, at a frontend of a storage platform, an incoming object having a first segment and a second segment different from the first segment, the second segment including a number of parallel segments processed by two or more object storage daemons (OSDs);
scan inline, by the processing device of the frontend, the first segment of the incoming object for a change of an entropy level of the first segment, wherein the scanning inline does not scan the second segment of the incoming object;
scan offline, subsequent to the receiving and by a backend of the storage platform, the second segment of the incoming object for a change of an entropy level of the second segment, the backend being more computationally powerful than the frontend, wherein the scanning offline of the second segment comprises scanning, by the backend of the storage platform, the second segment of the incoming object during processing of the incoming object for storage at the backend of the storage platform; and
determine an encryption of the incoming object when the change of the entropy level of the first segment or the second segment exceeds a threshold value based on the scanning inline of the first segment and the scanning offline of the second segment.
14 . The system of claim 13 , wherein the change of the entropy level of the first segment and the change of the entropy level of the second segment indicates an encryption by a ransomware to the incoming object.
15 . The system of claim 13 , the processing device and the memory further to:
generate a warning upon detecting the change of the entropy level of the first segment exceeding the threshold value indicating a ransomware encryption; and
generate a metadata record of the scanning offline of the second segment, regardless whether the change of the entropy level of the second segment exceeds the threshold value.
16 . The system of claim 15 , the processing device and the memory further to:
responsive to detecting that the change of the entropy level of the second segment exceeds the threshold value, provide the generated metadata record to an external policy engine for verification of the ransomware encryption.
17 . The system of claim 13 , wherein the first segment comprises an initial subset of the incoming object.