Synchronizable hardware security module

1. A computer-implemented method, comprising:

determining that a hardware security module in a hardware security module cluster contains one or more cryptographic keys that are not synchronized with at least one other hardware security module of the hardware security module cluster;

producing a set of key maps by acquiring, for a subset of hardware security modules in the hardware security module cluster, a key map that indicates key names and key versions retained on the subset of hardware security modules;

generating an update map from the set of key maps;

submitting the update map to the subset of hardware security modules in the hardware security module cluster;

identifying, based at least in part on the update map, a set of cryptographic keys not stored in the hardware security module;

receiving a set of encrypted cryptographic keys, a subset of cryptographic keys in the set of encrypted cryptographic keys encrypted using a cryptographic key available to the hardware security module in the hardware security module cluster, where the set of encrypted cryptographic keys includes at least one cryptographic key of the set of cryptographic keys not stored in the hardware security module;

generating an update request for the hardware security module in the hardware security module cluster, the update request including at least one encrypted cryptographic key selected from the set of encrypted cryptographic keys based at least in part on the key map of the hardware security module and the update map;

causing the hardware security module to update cryptographic-key information retained on the hardware security module by providing the update request to the hardware security module; and

causing the subset of hardware security modules in the hardware security module cluster to become synchronized by generating and providing additional update requests to the subset of hardware security modules.

2. The computer-implemented method of claim 1 , wherein determining that the hardware security module in the hardware security module cluster contains one or more cryptographic keys that are not synchronized with at least one other hardware security module by at least:

obtaining a set of data checksums including a checksum of contents of the subset of hardware security modules of the hardware security module cluster; and

determining that at least two checksums of the set of data checksums do not match.

3. The computer-implemented method of claim 1 , wherein the update map is generated in part by:

removing synchronized cryptographic keys from the set of key maps;

removing out-of-date cryptographic keys from the set of key maps; and

merging the set of key maps to form the update map.

4. The computer-implemented method of claim 1 , further comprising:

acquiring a write-lock by preventing the subset of hardware security modules in the hardware security module cluster from modifying cryptographic keys retained on the subset of hardware security modules;

determining that the hardware security module cluster is not synchronized;

causing the hardware security module cluster to become synchronized; and

releasing the write-lock by enabling the subset of hardware security modules in the hardware security module cluster to add or modify cryptographic keys retained on the subset of hardware security modules.

5. A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services:

acquire a first key map of a set of key maps from a first hardware security module that is a member of a hardware security module cluster, the first key map identifying a first set of cryptographic keys retained on the first hardware security module, where the set of key maps is generated based at least in part on information associated with cryptographic keys maintained by one or more hardware security modules of the hardware security module cluster;

acquire a second key map of the set of key maps from a second hardware security module that is a member of the hardware security module cluster, the second key map identifying a second set of cryptographic keys retained on the second hardware security module;

identify, based at least in part on the first key map and the second key map, a first cryptographic key stored on the first hardware security module that corresponds to an older version of a second cryptographic key on the second hardware security module;

acquire an encrypted version of the second cryptographic key from the second hardware security module, the encrypted version of the second cryptographic key encrypted with a cryptographic key that resides on both the first hardware security module and the second hardware security module; and

cause the first hardware security module to update the first cryptographic key to a value of the second cryptographic key by sending the encrypted version of the second cryptographic key to the first hardware security module.

6. The system of claim 5 , wherein the one or more services further:

acquire a third key map from a third hardware security module that is a member of a hardware security module cluster;

identify, based at least in part on the first key map and the third key map, a missing cryptographic key that is not stored on the first hardware security module and that is present on the third hardware security module;

acquire an encrypted version of the missing cryptographic key from the third hardware security module, the encrypted version of the missing cryptographic keys encrypted with a cryptographic key that resides on both the first hardware security module and the third hardware security module; and

store the missing cryptographic key on the first hardware security module by providing the encrypted version of the missing cryptographic key to the first hardware security module.

7. The system of claim 5 , wherein the one or more services:

acquire a first hash of a first hardware security module's contents;

acquire a third hash of a third hardware security module's contents;

determine that the first hash matches the third hash; and

cause the third hardware security module to update the first cryptographic key to a value of the second cryptographic key by sending the encrypted version of the second cryptographic key to the third hardware security module.

8. The system of claim 5 , wherein the one or more services further:

generate an update map based at least in part on the first key map and the second key map, the update map identifying a set of cryptographic keys that are not synchronized across the hardware security module cluster, the update map identifying a latest version of a subset of cryptographic keys in the set of cryptographic keys, and the update map identifying at least one hardware security module where a cryptographic key of the subset of cryptographic keys is stored; and

send the update map to the second hardware security module, causing the second hardware security module to provide the encrypted version of the cryptographic key to the one or more services.

9. The system of claim 5 , wherein:

the one or more services communicate with the first hardware security module via a first hardware security module cluster server that relays messages between the first hardware security module and the one or more services; and

the one or more services communicate with the second hardware security module via a second hardware security module cluster server that relays messages between the second hardware security module and the one or more services.

10. The system of claim 5 , wherein the one or more services further:

acquire a hash of contents for a portion of the hardware security modules in the hardware security module cluster;

identify a number of hardware security module groups containing one or more hardware security modules based at least in part on a set of associated hashes of contents of the one or more hardware security modules of the number of hardware security module groups;

identify a set of representative hardware security modules by at least selecting a representative hardware security module for at least one hardware security module group of the number of hardware security module groups;

generate an update for the set of representative hardware security modules, the update containing a set of encrypted cryptographic keys that, when transmitted to members of the set of representative hardware security modules causes a representative hardware security module receiving the update to become synchronized with a current state of the hardware security module cluster; and

send the update to the number of hardware security module groups.

11. The system of claim 10 , wherein:

the set of encrypted cryptographic keys is identified based on a set of key maps acquired from the set of representative hardware security modules; and

the set of encrypted cryptographic keys is acquired from a set of representative hardware security modules.

12. The system of claim 11 , wherein a particular representative hardware security module for a particular hardware security module group is selected based at least in part on a utilization measure determined for members of the particular hardware security module group.

13. The system of claim 10 , wherein the set of associated hashes of contents of the one or more hardware security modules is determined at least in part a cryptographic hash of cryptographic-key values and key names retained on the set of associated hashes of contents of the one or more hardware security modules.

14. The system of claim 5 , further comprising:

identify, based at least in part on the first key map and the second key map, a second particular cryptographic key stored on the second hardware security module that is an older version of an associated cryptographic key on the first hardware security module;

acquire an encrypted version of the associated cryptographic key from the first hardware security module, the encrypted version of the associated cryptographic key encrypted with the cryptographic key that resides on both the first hardware security module and the second hardware security module; and

cause the second hardware security module to update the second particular cryptographic key to a value of the associated cryptographic key by sending the encrypted version of the associated cryptographic key to the second hardware security module.

15. The system of claim 5 , further comprising:

generating an updated hash of contents of the first hardware security module;

providing the updated hash to the hardware security module cluster; and

providing an indication that the hardware security module cluster is synchronized.

16. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to at least:

collect a set of cryptographic-key inventories by acquiring an inventory of retained cryptographic keys and associated cryptographic-key versions for hardware security modules in a hardware security module cluster;

use the set of cryptographic-key inventories, identify a most recent version of one or more cryptographic keys retained on the hardware security module cluster;

identify an unsynchronized hardware security module that contains at least one outdated cryptographic key;

acquire an updated version of the outdated cryptographic key from a subset of hardware security modules in the hardware security module cluster; and

transmit an update, including the updated version of the outdated cryptographic key, to the unsynchronized hardware security module, the update causing the unsynchronized hardware security module to be placed in a synchronized state with the hardware security module cluster.

17. The non-transitory computer-readable storage medium of claim 16 , wherein:

a particular update sent to a particular unsynchronized hardware security module includes the updated version of the outdated cryptographic key on the particular unsynchronized hardware security module; and

the particular update sent to the particular unsynchronized hardware security module does not include a cryptographic key that does not correspond to at least one outdated cryptographic key on the particular unsynchronized hardware security module.

18. The non-transitory computer-readable storage medium of claim 16 , wherein a particular update sent to a particular unsynchronized hardware security module includes at least one cryptographic key that is not present on the particular unsynchronized hardware security module.

19. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions further comprise instructions that, as a result of being executed by the one or more processors, cause the computer system to:

before collecting the set of cryptographic-key inventories, acquiring a transactional lock on the hardware security module cluster, the transactional lock preventing other hardware security module cluster clients from modifying contents of the hardware security module cluster; and

after sending the one or more updates, releasing the transactional lock on the hardware security module cluster.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the updated version of the outdated cryptographic key is acquired by receiving an encrypted version of the update version encrypted with a cryptographic key available to at least one hardware security module in the hardware security module cluster.

21. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions that cause the computer system to acquire the updated version of the outdated cryptographic keys further include instructions that cause the computer system to:

provide a list identifying a set of most recent versions of a set of cryptographic keys retained on the hardware security module cluster to hardware security modules in the hardware security module cluster; and

receive, from hardware security modules in the hardware security module cluster, a set of cryptographic keys that are represented in the list.

22. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions that cause the computer system to acquire the updated version of the outdated cryptographic key further include instructions that cause the computer system to:

identify a particular hardware security module from the hardware security module cluster that is capable of providing at least one additional update version to at least one other outdate cryptographic key; and

acquire the at least one additional updated version possible from the particular hardware security module.