Systems and methods for improving asset download time using forward error correction
View Patent ↗The systems and methods described herein provide techniques for improving asset download time using forward error correction to recover lost packets or data. In various implementations, a sending device may be configured to encode redundant data in a data stream that enables a receiving device to reconstruct the data in the file if a portion of the data is not received. The redundant data may be based on single parity, in which the redundant data is generated using XOR (similar to the RAID-5 encoding methods), or the redundant data may be based on dual parity, in which the redundant data is generated using Galois fields (similar to the RAID-6 encoding methods). In some embodiments, the redundant data may be transmitted in separate packets. In other embodiments, the redundant data may be included as part of the data stream.
1 . A computer-implemented method of improving asset download time using forward error correction to recover lost packets or data, the method comprising:
splitting, by a sending device, data in a file to be transmitted into one or more data chunks;
encoding, by the sending device, a data stream comprising one or more data chunks with redundant data, wherein the redundant data is based on dual parity, and wherein the redundant data enables a receiving device to reconstruct the data in the file if a portion of the one or more data chunks is not received; and
transmitting, by the sending device to a receiving device, the encoded data stream.
2 . The computer-implemented method of claim 1 , wherein when based on dual parity, the redundant data encoded with a data chunk is generated based on Galois fields.
3 . The computer-implemented method of claim 1 , the method further comprising:
splitting, by the sending device, the file to be transmitted into a plurality of packets, wherein each of the one or more data chunks comprises a same number of the plurality of packets,
wherein encoding a data stream comprising one or more data chunks with redundant data comprises encoding each of the one or more data chunks with redundant data.
4 . The computer-implemented method of claim 3 , wherein the number of packets in each of the one or more data chunks is based on maximum expected packet loss.
5 . The computer-implemented method of claim 3 , wherein encoding each of the one or more data chunks with redundant data comprises generating two additional packets including the redundant data, wherein the two additional packets are generated based on Galois fields.
6 . The computer-implemented method of claim 1 , wherein encoding the data stream with redundant data comprises encoding redundant data after each of the one or more data chunks.
7 . The computer-implemented method of claim 6 , wherein the redundant data encoded after each data chunk is a size of an expected maximum of maximum transport unit (MTU).
8 . The computer-implemented method of claim 7 , wherein a desirable MTU is negotiated as part of an existing protocol.
9 . The computer-implemented method of claim 1 , the method further comprising:
receiving, by the receiving device, the transmitted data from the sending device, wherein the redundant data is received as part of the data stream.
10 . The computer-implemented method of claim 9 , wherein the data stream is transmitted using an existing protocol by a standard sending device.
11 . The computer-implemented method of claim 9 , wherein the receiving device receives the data stream using an existing protocol.
12 . The computer-implemented method of claim 11 , wherein the existing protocol is QUIC.
13 . The computer-implemented method of claim 11 , wherein the existing protocol is HTTP/3.
14 . The computer-implemented method of claim 1 , the method further comprising:
receiving, by the receiving device, the encoded data stream;
determining, by the receiving device, that the received data stream is missing a portion of the one or more data chunks;
responsive to the determination that the received data stream is missing a portion of the one or more data chunks, reconstructing the missing portion of the one or more data chunks using the redundant data; and
providing, by the receiving device, the received data stream to an application layer with the reconstructed portion.
15 . A system for improving asset download time using forward error correction to recover lost packets or data, the system comprising:
one or more processors configured by computer readable instructions to:
split data in a file to be transmitted into one or more data chunks;
encode a data stream comprising one or more data chunks with redundant data, wherein the redundant data is based on dual parity, and wherein the redundant data enables a receiving device to reconstruct the data in the file if a portion of the one or more data chunks is not received; and
transmit the encoded data stream to a receiving device.
16 . The system of claim 15 , wherein when based on dual parity, the redundant data encoded with a data chunk is generated based on Galois fields.
17 . The system of claim 15 , wherein the one or more processors are further configured to:
split the file to be transmitted into a plurality of packets, wherein each of the one or more data chunks comprises a same number of the plurality of packets,
wherein to encode a data stream comprising one or more data chunks with redundant data, the one or more processors are configured to encode each of the one or more data chunks with redundant data.
18 . The system of claim 17 , wherein the number of packets in each of the one or more data chunks is based on maximum expected packet loss.
19 . The system of claim 17 , wherein to encode each of the one or more data chunks with redundant data, the one or more processors are configured to generate two additional packets including the redundant data, wherein the two additional packets are generated based on Galois fields.
20 . The system of claim 15 , wherein to encode the data stream with redundant data, the one or more processors are configured to encode redundant data after each of the one or more data chunks.
21 . The system of claim 20 , wherein the redundant data encoded after each data chunk is a size of an expected maximum of maximum transport unit (MTU).
22 . The system of claim 21 , wherein a desirable MTU is negotiated as part of an existing protocol.
23 . The system of claim 15 , wherein the receiving device receives the transmitted data, and wherein the redundant data is received as part of the data stream.
24 . The system of claim 23 , wherein the data stream is transmitted using an existing protocol by a standard sending device.
25 . The system of claim 23 , wherein the receiving device receives the data stream using an existing protocol.
26 . The system of claim 25 , wherein the existing protocol is QUIC.
27 . The system of claim 25 , wherein the existing protocol is HTTP/3.
28 . The system of claim 15 , wherein the receiving device is configured to receive the encoded data stream, determine that the received data stream is missing a portion of the one or more data chunks, reconstruct the missing portion of the one or more data chunks using the redundant data responsive to the determination that the received data stream is missing a portion of the one or more data chunks, and provide the received data stream to an application layer with the reconstructed portion.