Fast distributed caching using erasure coded object parts
Systems and methods are described for providing rapid access to data objects stored in a cache. Rather than storing data objects directly, each object can be broken into a number of parts via erasure coding, which enables the object to be generated from less than all parts. When servicing a request for the data object, a device can attempt to retrieve all parts, but begin to generate the data object as soon as a sufficient number of parts is retrieved, even if requests for other parts are outstanding. In this way, the data object can be retrieved without delay due to the slowest requests. For example, where one or more requests timeout, such as due to failure of cache devices, this timeout may have no effect on time required to retrieve the data object from the cache.
1. A system for facilitating retrieval of data objects from a cache storing the data objects as sets of erasure coded parts, the system comprising:
a plurality of cache devices, each cache device including a cache configured to store erasure coded parts; and
one or more processors configured to execute computer-executables instructions to:
obtain a request for a data object;
identify, according to at least an erasure coding scheme corresponding to the data object, a set of cache devices, from the plurality of cache devices, assigned to store a set of erasure coded parts that represent the data object, wherein each cache device of the set of cache devices is assigned to store a corresponding erasure coded part of the set of erasure coded parts;
request, from the set of cache devices, the set of erasure coded parts;
retrieve, from the set of cache devices, a threshold number of erasure coded parts, from the set of erasure coded parts, that is sufficient to generate the data object, the threshold number being less than all of the set of erasure coded parts;
when the threshold number of erasure coded parts is retrieved and while at least one request for an erasure coded part of the set is outstanding, generate the data object from the threshold number of erasure coded parts;
return the data object in response to the request;
obtain a request for a second data object;
identify, according to at least the erasure coding scheme, a second set of cache devices assigned to store a second set of erasure coded parts that represent the second data object;
determine that the second set of cache devices lack a sufficient number of the second set of erasure coded parts to generate the data object; and
retrieve the data object from an origin location for the data object.
2. The system of claim 1 , wherein the erasure coding scheme establishes the threshold number as either one less or two less than a total number of parts within the set of erasure coded parts.
3. The system of claim 1 , wherein the one or more processors are further configured to:
identify at least one erasure coded part of the set of erasure coded parts that is not stored within the cache device of the set of cache devices that is assigned to store the at least one erasure coded part;
erasure code the data object to result in the at least one erasure coded part; and
store the at least one erasure coded part in the cache device.
4. The system of claim 1 , wherein the plurality of cache devices are arranged into a consistent hash ring, and wherein, to identify the set of cache devices assigned to store the set of erasure coded parts, the computer-executables instructions cause the one or more processors to:
determine a hash value for the data object; and
identify the set of cache devices based at least on the hash value for the object and locations of the set of cache devices on the consistent hash ring.
5. The system of claim 4 , wherein the set of cache devices are identified as a first n cache device on the consistent hash ring at locations subsequent to the hash value of the data object, where n equals the number of cache devices within the set of cache devices.
6. A computer-implemented method comprising:
obtaining a request for a data object;
identifying, according to at least an erasure coding scheme corresponding to the data object, a set of cache devices assigned to store a set of erasure coded parts that represent the data object, wherein each cache device of the set of cache devices is assigned to store a corresponding erasure coded part of the set of erasure coded parts;
requesting, from the set of cache devices, the set of erasure coded parts;
retrieving, from the set of cache devices, a threshold number of erasure coded parts, from the set of erasure coded parts, that is sufficient to generate the data object, the threshold number being less than all of the set of erasure coded parts;
when the threshold number of erasure coded parts is retrieved and while at least one request for an erasure coded part of the set is outstanding, initiating generation of the data object from the threshold number of erasure coded parts;
returning the data object in response to the request;
obtaining a request for a second data object;
identifying, according to at least the erasure coding scheme, a second set of cache devices assigned to store a second set of erasure coded parts that represent the second data object;
determining that the second set of cache devices lack a sufficient number of the second set of erasure coded parts to generate the data object; and
retrieving the data object from an origin location for the data object.
7. The computer-implemented method of claim 6 , wherein the origin location is identified within the request for the second data object.
8. The computer-implemented method of claim 6 , wherein retrieving the data object from the origin location for the data object begins while at least one request to a cache device of the second set of cache devices for an erasure coded part, of the set of erasure coded parts, is outstanding.
9. The computer-implemented method of claim 6 further comprising:
erasure coding the data object into the set of erasure coded parts;
determine one or more erasure coded parts, of the set of erasure coded parts, not stored on the set of cache devices; and
for each erasure coded part, of the set of erasure coded parts, determined not to be stored by the set of cache devices, storing the erasure coded part at the set of cache devices.
10. The computer-implemented method of claim 6 , wherein the set of cache devices are arranged into a consistent hash ring comprising a plurality of cache devices, and wherein identifying the set of cache devices assigned to store the set of erasure coded parts comprises:
determining a hash value for the data object; and
identifying the set of cache devices based at least on the hash value for the object and locations of the set of cache devices on the consistent hash ring.
11. The computer-implemented method of claim 10 , wherein the hash value is an identifier of the data object.
12. The computer-implemented method of claim 10 , wherein determining the hash value for the data object comprises passing an identifier of the data object through a hash function.
13. Non-transitory computer-readable media including computer-executable instructions that, when executed by a computing system, cause the computing system to:
obtain a request for a data object;
identify, according to at least an erasure coding scheme corresponding to the data object, a set of cache devices assigned to store a set of erasure coded parts that represent the data object, wherein each cache device of the set of cache devices is assigned to store a corresponding erasure coded part of the set of erasure coded parts;
request, from the set of cache devices, the set of erasure coded parts;
retrieve, from the set of cache devices, a threshold number of erasure coded parts, from the set of erasure coded parts, that is sufficient to generate the data object, the threshold number being less than all of the set of erasure coded parts;
when the threshold number of erasure coded parts is retrieved and while at least one request for an erasure coded part of the set is outstanding, initiate generation of the data object from the threshold number of erasure coded parts;
return the data object in response to the request;
obtain a request for a second data object;
identify, according to at least the erasure coding scheme, a second set of cache devices assigned to store a second set of erasure coded parts that represent the second data object;
determine that the second set of cache devices lack a sufficient number of the second set of erasure coded parts to generate the second data object; and
retrieve the second data object from an origin location for the data object.
14. The non-transitory computer-readable media of claim 13 , wherein the computer-executable instructions further cause the computing system to store the data object in a local cache of the computing system, the local cache representing a level one cache and the set of cache devices representing at least part of a level two cache.
15. The non-transitory computer-readable media of claim 13 , wherein the computer-executable instructions further cause the computing system to:
identify at least one erasure coded part of the set of erasure coded parts that is not stored within the cache device of the set of cache devices that is assigned to store the at least one erasure coded part;
erasure code the data object to result in the at least one erasure coded part; and
store the at least one erasure coded part in the cache device.
16. The non-transitory computer-readable media of claim 13 , wherein the computer-executable instructions cause the computing system to initiate retrieval of the second data object from the origin location for the second data object while at least one request to a cache device of the second set of cache devices for an erasure coded part, of the second set of erasure coded parts, is outstanding.
17. The non-transitory computer-readable media of claim 13 , wherein the computer-executable instructions further cause the computing system to:
erasure code the data object into the set of erasure coded parts;
determine one or more erasure coded parts, of the set of erasure coded parts, not stored on the set of cache devices; and
for each erasure coded part, of the set of erasure coded parts, determined not to be stored by the set of cache devices, store the erasure coded part at the set of cache devices.
18. The non-transitory computer-readable media of claim 13 , wherein the set of cache devices are arranged into a consistent hash ring comprising a plurality of cache devices, and wherein to identify the set of cache devices assigned to store the set of erasure coded parts, the computer-executable instructions cause the computing system to:
determine a hash value for the data object; and
identify the set of cache devices based at least on the hash value for the object and locations of the set of cache devices on the consistent hash ring.
19. The non-transitory computer-readable media of claim 13 , wherein to request the set of erasure coded parts from the set of cache devices, the computer-executable instructions cause the computing system to:
request a subset of the set of erasure coded parts from the cache devices, of the set of cache devices, expected to store the subset;
detect that at least one response from the cache devices has not been received in a threshold period of time; and
request a remainder of the set of erasure coded parts from remaining cache devices of the set of cache devices.