Repeated collections of vulnerability assessment data from remote machine
Systems and methods are disclosed to implement a delta data collection technique for collecting machine characteristics data from client machines. In embodiments, the collected data is used by a machine assessment service to maintain a virtual representation of the client machine for assessments. To initialize the collection process, the client uploads an initial copy of the data in full. Subsequently, the client determines periodic deltas between a current baseline of the data and a last reported baseline, and the deltas are uploaded as patches. The machine assessment service then applies these patches to update the virtual representation of the client machine. In embodiments, to facilitate the generation or uploading of the patches, the client may generate the baselines in a different encoding format as used by the data. For example, baselines in the new encoding format may be more easily compared and manipulated during the patch generation process.
1. A system, comprising:
one or more hardware processors with associated memory that implement a machine assessment service, configured to:
configure, at a client machine, an agent to collect machine characteristics data about the client machine;
receive, from the agent, an initial baseline of the machine characteristics data about the client machine, wherein the initial baseline specifies a plurality of keys or key-value pairs in a binary encoding format;
create a snapshot of the machine characteristics data in the binary encoding format using the initial baseline;
at one or more times subsequent to the creation of the snapshot:
receive a patch of the snapshot from the agent, wherein the patch is in a text encoding format with keys sorted in an alphanumeric order,
apply the patch to the snapshot to update the snapshot to the current state of the machine characteristics data at the client machine,
generate a check value by applying the one or more hash functions to the updated snapshot, and
validate the updated snapshot based at least in part on a comparison of the check value with another check value received from the agent; and
use the updated snapshot to assess the client machine for one or more security vulnerabilities.
2. The system of claim 1 , wherein the machine assessment service is implemented using virtualized compute resources hosted by a service provider network.
3. The system of claim 1 , wherein the machine assessment service is configured to monitor a plurality of client machines for security vulnerabilities, including virtual machines hosted on virtualization hosts.
4. The system of claim 1 , wherein the snapshot is a static registry hive that contains at least a portion of data stored in an active registry hive of the client machine.
5. The system of claim 4 , wherein to apply the patch to the snapshot, the machine assessment service is configured to:
determine, from the patch file, a new entry to be created in the static registry hive;
allocate a memory location in the static registry hive for the new entry, wherein the memory location holds obsolete data and has sufficient size to store the new entry; and
reuse an obsolete memory location to store the new entry.
6. The system of claim 1 , wherein the machine assessment service is configured to:
receive, from the agent, patches for multiple different categories of machine characteristics data; and
store multiple snapshots of the multiple different categories of machine characteristics data as part of a virtual representation of the client machine.
7. The system of claim 1 , wherein the machine assessment service is configured to store a sequence of patches received from the agent in an order that the patches were applied to the snapshot.
8. The system of claim 7 , wherein the machine assessment service is configured to analyze the sequence of patches to determine one or more metrics of the client machine, comprising one or more of:
a frequency of change of the machine characteristics data of the client machine; or
a frequency of snapshot validation errors for patches received from the client machine.
9. The system of claim 7 , wherein the machine assessment service is configured to:
determine, based at least in part on the sequence of patches and a new patch received from the agent, that the client machine is a clone of another client machine that has sent patches to the machine assessment service; and
responsive to the determination that the client machine is a clone, send a message instructing the agent to use a new identifier for uploading further patches.
10. The system of claim 7 , wherein the machine assessment service is configured to:
determine that a new patch received from the agent is not valid with respect to a current snapshot state of the client machine;
determine that the new patch is valid with respect to a former snapshot state of the client machine indicated by the sequence of patches; and
responsive to the determination that the new patch is valid with respect to the former snapshot state, apply the new patch to the former snapshot state to create a new branch in the sequence of patches for the client machine, without requesting a new baseline to be collected from the client machine.
11. A method, comprising:
performing, by a machine assessment service implemented by one or more hardware processors with associated memory:
configuring, at a client machine, an agent to collect machine characteristics data about the client machine;
receiving, from the agent, an initial baseline of the machine characteristics data about the client machine, wherein the initial baseline specifies a plurality of keys or key-value pairs in a binary encoding format;
creating a snapshot of the machine characteristics data in the binary encoding format using the initial baseline; and
at one or more times subsequent to the creation of the snapshot:
receiving a patch of the snapshot from the agent, wherein the patch is in a text encoding format with keys sorted in an alphanumeric order,
applying the patch to the snapshot to update the snapshot to the current state of the machine characteristics data at the client machine,
generating a check value by applying the one or more hash functions to the updated snapshot, and
validating the updated snapshot based at least in part on a comparison of the check value with another check value received from the agent; and
using the updated snapshot to assess the client machine for one or more security vulnerabilities.
12. The method of claim 11 , further comprising the machine assessment service receiving patches periodically from the agent.
13. The method of claim 11 , further comprising the machine assessment service sending data collection instruction to the agent, wherein the agent is configured to collect machine characteristics data according to the data collection instructions.
14. The method of claim 11 , further comprising the machine assessment service:
determining that the snapshot is not valid after applying a second patch, and in response:
rejecting application of the second patch; and
instructing the agent to upload a full baseline of the machine characteristics data.
15. The method of claim 11 , further comprising the machine assessment service receiving an upload event associated with the patch, wherein the upload load event indicates a location where the patch is uploaded and a description of contents of the patch.
16. The method of claim 15 , further comprising the machine assessment service initiating a snapshot update job in response to the upload event to apply the patch at the location.
17. The method of claim 15 , wherein the check value is a hash value computed from the snapshot, and the other check value is indicated in the upload event.
18. The method of claim 15 , wherein the upload event indicates a previous patch uploaded by the agent.
19. The method of claim 11 , further comprising the machine assessment:
storing a sequence of patches received from the agent in an order that the patches were applied to the snapshot;
determining, based at least in part on the sequence of patches and a new patch received from the agent, that the client machine is a clone of another client machine that has sent patches to the machine assessment service; and
responsive to the determination that the client machine is a clone, sending a message instructing the agent to use a new identifier for uploading further patches.
20. The method of claim 11 , further comprising the machine assessment:
storing a sequence of patches received from the agent in an order that the patches were applied to the snapshot;
determining that a new patch received from the agent is not valid with respect to a current snapshot state of the client machine;
determining that the new patch is valid with respect to a former snapshot state of the client machine indicated by the sequence of patches; and
responsive to the determination that the new patch is valid with respect to the former snapshot state, applying the new patch to the former snapshot state to create a new branch in the sequence of patches for the client machine, without requesting a new baseline to be collected from the client machine.