Backup server selection based on data commonality
Techniques of backing up data stored on host computing devices involve selecting a backup server from among multiple servers on which to back up host data based on a measure of commonality between the host data and data stored in the backup servers. Prior to sending data for backup, a host sends a set of host data representations to a backup system. Each host data representation is based on a respective hash value computed from a respective block of the host data. The backup system compares the set of host data representations with server data representations for each backup server and computes a commonality score for each backup server. The backup system then selects a backup server on which to place the host data based at least in part on the commonality scores. Host data are then directed to the selected backup server for backup.
1. A method of backing up data stored on host computing devices, the method comprising:
receiving, by processing circuitry of a backup system, (i) a request to back up host data stored on a host computing device and (ii) a set of multiple host data representations, each host data representation based on a respective hash value computed from a respective block of the host data of the host computing device;
computing multiple commonality scores, including one commonality score for each of multiple backup servers in the backup system, each commonality score for a respective backup server indicating a measure of commonality between the set of host data representations and a respective set of server data representations for the respective backup server, each backup server including a respective set of storage devices; and
backing up the host data by (i) selecting one of the backup servers on which to back up the host data based at least in part on the commonality scores, and (ii) storing the host data in the set of storage devices of the selected backup server,
wherein backing up the host data further includes performing a load balancing assessment, the load balancing assessment producing a load balancing result indicating relative loading of the backup servers, and wherein selecting one of the backup servers on which to back up the host data is also based in part on the load balancing result.
2. A method as in claim 1 , wherein selecting the backup server on which to back up the host data includes (i) comparing the commonality scores to identify a backup server having the highest commonality score and (ii) identifying the selected backup server as the backup server having the highest commonality score.
3. A method as in claim 2 , wherein receiving the set of multiple host data representations includes receiving, as each data representation, the respective hash value,
wherein each hash value was computed by applying a cryptographic hash function to the respective block of host data,
wherein each of the backup servers stores a respective set of data blocks, and
wherein the method further comprises, for each backup server, generating the set of server data representations for the respective backup server by applying the cryptographic hash function to each of the set of data blocks of the respective backup server to produce, as the set of server data representations for the respective backup server, a set of hash values for the respective backup server.
4. A method as in claim 3 , wherein computing the multiple commonality scores includes, for each of the backup servers, counting a number of matches between the hash values received from the host and the set of hash values for the respective backup server to produce a total for the respective backup server, wherein the selected backup server is the backup server for which the largest total is produced.
5. A method as in claim 4 , wherein each hash value is M bits in length, and wherein applying the cryptographic function to each of the set of data blocks of each backup server includes (i) applying a hash function to generate a N-bit result and (ii) sampling the N-bit result to produce an M-bit result, wherein M is less than N.
6. A method as in claim 2 , wherein the set of host data representations includes a bloom filter of blocks of the host data, the bloom filter including (i) a set of cryptographic hash functions and (ii) a bit string of a predetermined length, each of the set of cryptographic hash functions mapping a block of host data to a respective position in the bit string, the bit string having a set of mapped positions and a set of unmapped positions, each mapped position in the bit string having a first value, each unmapped position in the bit string having a second value;
wherein each of the backup servers stores respective blocks of data;
wherein the method further comprises, for each of the backup servers, applying each of the set of cryptographic hash functions to a block of data of the respective backup server to produce a bit position of that hash function for that block of data;
wherein computing the multiple commonality scores includes, for each of the backup servers, (i) applying each of the cryptographic hash functions to a block of data of the respective backup server to produce a set of bit positions and (ii) reducing the commonality score for the respective backup server in response to at least one of the produced set of bit positions of the bit string of the bloom filter having the second value.
7. A method as in claim 1 , wherein the host data representations received by the backup system pertain to a subset of all of the data blocks stored in the host computing device.
8. A method as in claim 7 , wherein each set of server data representations for a backup server pertains to a subset of all of the data blocks stored on the respective backup server.
9. A method as in claim 1 , wherein each set of server data representations for a particular backup server pertains to a subset of all of the data blocks stored on the particular backup server.
10. A computer program product including a set of non-transitory, computer-readable storage media storing executable instructions, which when executed by a computer, causes the computer to perform a method of backing up data stored on host computing devices, the method comprising:
receiving (i) a request to back up host data stored on a host computing device and (ii) a set of multiple host data representations, each host data representation based on a respective hash value computed from a respective block of the host data of the host computing device;
computing multiple commonality scores, including one commonality score for each of multiple backup servers in the backup system, each commonality score for a respective backup server indicating a measure of commonality between the set of host data representations and a respective set of server data representations for the respective backup server, each backup server including a respective set of storage devices; and
backing up the host data by (i) selecting one of the backup servers on which to back up the host data based at least in part on the commonality scores, and (ii) storing the host data in the set of storage devices of the selected backup server,
wherein selecting the backup server on which to back up the host data includes (i) comparing the commonality scores to identify a backup server having the highest commonality score and (ii) identifying the selected backup server as the backup server having the highest commonality score,
wherein receiving the set of multiple host data representations includes receiving, as each data representation, the respective hash value,
wherein each hash value was computed by applying a cryptographic hash function to the respective block of host data,
wherein each of the backup servers stores a respective set of data blocks,
wherein the method further comprises, for each backup server, generating the set of server data representations for the respective backup server by applying the cryptographic hash function to each of the set of data blocks of the respective backup server to produce, as the set of server data representations for the respective backup server, a set of hash values for the respective backup server,
wherein computing the multiple commonality scores includes, for each of the backup servers, counting a number of matches between the hash values received from the host and the set of hash values for the respective backup server to produce a total for the respective backup server, and
wherein the selected backup server is the backup server for which the largest total is produced.
11. A computer program product as in claim 10 , each hash value is M bits in length, and wherein applying the cryptographic function to each of the set of data blocks of each backup server includes (i) applying a hash function to generate a N-bit result and (ii) sampling the N-bit result to produce an M-bit result, wherein M is less than N.
12. A computer program product as in claim 10 , wherein the set of host data representations includes a bloom filter of blocks of the host data, the bloom filter including (i) a set of cryptographic hash functions and (ii) a bit string of a predetermined length, each of the set of cryptographic hash functions mapping a block of host data to a respective position in the bit string, the bit string having a set of mapped positions and a set of unmapped positions, each mapped position in the bit string having a first value, each unmapped position in the bit string having a second value;
wherein each of the backup servers stores respective blocks of data;
wherein the method further comprises, for each of the backup servers, applying each of the set of cryptographic hash functions to a block of data of the respective backup server to produce a bit position of that hash function for that block of data;
wherein computing the multiple commonality scores includes, for each of the backup servers, (i) applying each of the cryptographic hash functions to a block of data of the respective backup server to produce a set of bit positions and (ii) reducing the commonality score for the respective backup server in response to at least one of the produced set of bit positions of the bit string of the bloom filter having the second value.
13. A computer program product as in claim 10 , wherein each set of server data representations for a backup server pertains to a subset of all of the data blocks stored on the respective backup server.
14. A computer program product as in claim 13 , wherein each set of server data representations for a backup server pertains to a subset of all of the data blocks stored on the respective backup server.
15. A computer program product as in claim 10 , wherein each set of server data representations for a particular backup server pertains to a subset of all of the data blocks stored on the particular backup server.
16. An electronic apparatus comprising memory and controlling circuitry coupled to the memory, the controlling circuitry being constructed and arranged to:
receive (i) a request to back up host data stored on a host computing device and (ii) a set of multiple host data representations, each host data representation based on a respective hash value computed from a respective block of the host data of the host computing device;
compute multiple commonality scores, including one commonality score for each of multiple backup servers in the backup system, each commonality score for a respective backup server indicating a measure of commonality between the set of host data representations and a respective set of server data representations for the respective backup server, each backup server including a respective set of storage devices; and
back up the host data by (i) selecting one of the backup servers on which to back up the host data based at least in part on the commonality scores, and (ii) storing the host data in the set of storage devices of the selected backup server,
wherein the controlling circuitry constructed and arranged to back up the host data further is further constructed and arranged to perform a load balancing assessment, the load balancing assessment producing a load balancing result indicating relative loading of the backup servers, and wherein selecting one of the backup servers on which to back up the host data is also based in part on the load balancing result.
17. An electronic apparatus as in claim 16 , wherein selecting the backup server on which to back up the host data includes (i) comparing the commonality scores to identify a backup server having the highest commonality score and (ii) identifying the selected backup server as the backup server having the highest commonality score.
18. An electronic apparatus as in claim 17 , wherein each of the set of host data representations is a respective hash value,
wherein each hash value is computed by applying a cryptographic hash function to the respective block of host data,
wherein each of the backup servers stores a respective set of data blocks, and
wherein the controlling circuitry is further constructed and arranged to generate, for each backup server, the set of server data representations for the respective backup server by applying the cryptographic hash function to each of the set of data blocks of the respective backup server to produce, as the set of server data representations for the respective backup server, a set of hash values for the respective backup server.
19. An electronic apparatus as in claim 18 , wherein the controlling circuitry constructed and arranged to compute the multiple commonality scores is further constructed and arranged to count, for each of the backup servers, a number of matches between the hash values received from the host and the set of hash values for the respective backup server to produce a total for the respective backup server, wherein the selected backup server is the backup server for which the largest total is produced.
20. An electronic apparatus as in claim 17 , wherein the set of host data representations includes a bloom filter of blocks of the host data, the bloom filter including (i) a set of cryptographic hash functions and (ii) a bit string of a predetermined length, each of the set of cryptographic hash functions mapping a block of host data to a respective position in the bit string, the bit string having a set of mapped positions and a set of unmapped positions, each mapped position in the bit string having a first value, each unmapped position in the bit string having a second value;
wherein each of the backup servers stores respective blocks of data;
wherein the controlling circuitry is further constructed and arranged to, for each of the backup servers, apply each of the set of cryptographic hash functions to a block of data of the respective backup server to produce a bit position of that hash function for that block of data;
wherein the controlling circuitry constructed and arranged to compute the multiple commonality scores is further constructed and arranged to, for each of the backup servers, (i) apply each of the cryptographic hash functions to a block of data of the respective backup server to produce a set of bit positions and (ii) reduce the commonality score for the respective backup server in response to at least one of the produced set of bit positions of the bit string of the bloom filter having the second value.