Reduction of execution stalls of LZ4 decompression via parallelization
A parallel decompression engine has separate hardware paths for sequence processing and repeated byte copying/placement. Literal byte extraction logic obtains literal bytes from a selected sequence. Literal byte write logic writes the obtained literal bytes into an uncompressed data set that is being generated. Offset and length extraction logic obtains the offset and length of repeated bytes from the selected sequence. In a separate hardware path, copying and placement logic uses the offset and length to find and copy the length of repeated bytes at the specified offset in the uncompressed data set, and place the copied repeated bytes back into the uncompressed data set adjacent to the literal bytes.
1. An apparatus comprising:
a parallel decompression engine that processes a compressed data set to generate a corresponding uncompressed data set, the parallel decompression engine comprising:
sequence processing logic comprising:
literal byte extraction logic that obtains literal bytes from a selected sequence of the compressed data set;
literal byte write logic that writes the obtained literal bytes into the uncompressed data set; and
offset and length extraction logic that obtains an offset and length of repeated bytes from the selected sequence; and
copying and placement logic that uses the offset and length to copy the length of repeated bytes at the offset in the uncompressed data set, and place the copied repeated bytes at a location in the uncompressed data set adjacent to the literal bytes of the selected sequence;
wherein the sequence processing logic is implemented by a first hardware path and the copying and placement logic is implemented by a second hardware path.
2. The apparatus of claim 1 comprising a first special purpose integrated circuit in which the sequence processing logic is implemented, and a second special purpose integrated circuit in which the copying and placement logic is implemented.
3. The apparatus of claim 1 comprising a special purpose integrated circuit in which the parallel decompression engine is implemented, the special purpose integrated circuit comprising a first execution unit in which the sequence processing logic is implemented, and a second execution unit in which the copying and placement logic is implemented.
4. The apparatus of claim 1 comprising a first processor core in which the sequence processing logic is implemented, and a second processor core in which the copying and placement logic is implemented.
5. The apparatus of claim 1 comprising a first group of allocated processor cores in which the sequence processing logic is implemented, and a second group of allocated processor cores in which the copying and placement logic is implemented.
6. The apparatus of claim 5 comprising pairings of individual processor cores of the first group with individual processor cores of the second group.
7. The apparatus of claim 1 comprising a special purpose integrated circuit in which the sequence processing logic is implemented, and a general purpose multi-core processor in which the copying and placement logic is implemented.
8. The apparatus of claim 1 comprising a general purpose multi-core processor in which the sequence processing logic is implemented, and a special purpose integrated circuit in which the copying and placement logic is implemented.
9. The apparatus of claim 1 comprising a computing node in which the parallel decompression engine is implemented.
10. The apparatus of claim 9 comprising a storage array in which the computing node is implemented.
11. A method comprising:
generating an uncompressed data set from a compressed data set, comprising:
sequence processing in a first hardware path, comprising:
obtaining literal bytes from a selected sequence of the compressed data set;
writing the obtained literal bytes into the uncompressed data set; and
obtaining an offset and length of repeated bytes from the selected sequence; and
copying and placement processing in a second hardware path, comprising:
using the offset and length for copying the length of repeated bytes at the offset in the uncompressed data set, and placing the copied repeated bytes at a location in the uncompressed data set adjacent to the literal bytes of the selected sequence.
12. The method of claim 11 comprising performing sequence processing with a first special purpose integrated circuit, and performing copying and placement processing with a second special purpose integrated circuit.
13. The method of claim 11 comprising performing parallel decompression with a special purpose integrated circuit, comprising performing sequence processing with a first execution unit of the special purpose integrated circuit, and performing copying and placement processing with a second execution unit of the special purpose integrated circuit.
14. The method of claim 11 comprising performing sequence processing with a first processor core, and performing copying and placement processing with a second processor core.
15. The method of claim 11 comprising performing sequence processing with a first group of allocated processor cores, and performing copying and placement processing with a second group of allocated processor cores.
16. The method of claim 15 comprising pairing individual processor cores of the first group with individual processor cores of the second group.
17. The method of claim 11 comprising performing sequence processing with a special purpose integrated circuit, and performing copying and placement processing with a general purpose multi-core processor.
18. The method of claim 11 comprising performing sequence processing with a general purpose multi-core processor, and performing copying and placement processing with a special purpose integrated circuit.
19. The method of claim 11 comprising operating the parallel decompression engine in a computing node.
20. The method of claim 19 comprising operating the parallel decompression engine in a storage array.