Integrated erasure coding for data storage and transmission
A network device includes memory with permanent storage. A request for information is received from a user device. In response to determining that an erasure encoded version of the requested information is not stored in the memory of the network device, the requested information is requested to be transmitted from a storage external to the network device and partially received. In response to determining that an amount of the partially received requested information does not satisfy an erasure encoding threshold, it is determined whether the erasure encoding threshold can be satisfied within a time duration. If so, the partially received requested information is erasure encoded, stored in the permanent storage, and transmitted. If not, the requested information is requested to be retransmitted from the storage external to the network device.
1. A network device, comprising:
memory; and
one or more processors configured to:
receive a request for information from a user device;
partially receive the requested information from a storage external to the network device; and
in response to determining that an amount of the partially received requested information does not satisfy an erasure encoding threshold:
in response to determining that the erasure encoding threshold can be satisfied within a time duration:
erasure encode the partially received requested information; and
transmit the erasure encoded partially received requested information, wherein the amount of the partially received requested information is a subset of information needed to reconstruct the requested information at the network device; and
in response to determining that the erasure encoding threshold cannot be satisfied within the time duration, request that the requested information be retransmitted from the storage external to the network device.
2. The network device of claim 1 , wherein the one or more processors are further configured to store the erasure encoded partially received requested information in a permanent storage of the memory.
3. The network device of claim 1 , wherein the one or more processors are further configured to:
request that the requested information be transmitted from a data center and receive the requested information from the data center;
determine if the received requested information can be stored in the memory; and
in response to determining that the received requested information cannot be stored in the memory, transmit the received requested information based on a network condition.
4. The network device of claim 3 , wherein transmission of the received requested information based on the network condition comprises transmitting the received requested information via network coding using a butterfly network with at least one multicast session.
5. The network device of claim 3 , wherein transmission of the received requested information based on the network condition comprises selecting a type of network coding based on the network condition, wherein the type of network coding includes at least one of a deterministic network coding scheme and a random network coding scheme.
6. The network device of claim 3 , wherein the one or more processors are further configured to:
in response to determining that the received requested information can be stored in the memory:
in response to determining that an amount of the received requested information satisfies an erasure encoding threshold, erasure encode the received requested information, store the erasure encoded received requested information in a permanent storage of the memory, and transmit the erasure encoded received requested information based on the network condition.
7. A method of transmitting data comprising:
partially receiving requested information from a storage external to a network device; and
in response to determining that an amount of the partially received requested information does not satisfy an erasure encoding threshold:
in response to determining that the erasure encoding threshold can be satisfied within a time duration:
erasure encoding the partially received requested information; and
transmitting the erasure encoded partially received requested information, wherein the amount of the partially received requested information is a subset of information needed to reconstruct the requested information at the network device; and
in response to determining that the erasure encoding threshold cannot be satisfied within the time duration, requesting that the requested information be retransmitted from the storage external to the network device.
8. The method of claim 7 , further comprising storing the erasure encoded partially received requested information in a permanent storage of the network device.
9. The method of claim 7 , further comprising:
requesting that the requested information be transmitted from a data center and receiving the requested information from the data center;
determining if the received requested information can be stored in a memory of the network device; and
in response to determining that the received requested information cannot be stored in the memory, transmitting the received requested information based on a network condition.
10. The method of claim 9 , wherein transmission of the received requested information based on the network condition comprises transmitting the received requested information via network coding using a butterfly network with at least one multicast session.
11. The method of claim 9 , wherein transmission of the received requested information based on the network condition comprises selecting a type of network coding based on the network condition, wherein the type of network coding includes at least one of a deterministic network coding scheme and a random network coding scheme.
12. The method of claim 9 , further comprising:
in response to determining that the received requested information can be stored in the memory of the network device:
in response to determining that an amount of the received requested information satisfies an erasure encoding threshold, erasure encoding the received requested information, storing the erasure encoded received requested information in a permanent storage of the network device, and transmitting the erasure encoded received requested information based on the network condition.
13. The method of claim 7 , wherein the requested information comprises video content.
14. The method of claim 7 , wherein transmitting the erasure encoded partially received requested information comprises selecting a type of network coding based on a network condition.