Txilm: lossy block compression with salted short hashing
Compression techniques by pre-sorting transactions in a consistent way. In at least one embodiment, the compression does not rely on consistent mempool across full nodes. Transactions in a block can be hashed and sorted. Ambiguity can arise from the hashes, which can be resolved using various techniques.
1. A computer-implemented method comprising:
sorting a plurality of transactions of a block of a blockchain network based at least in part on a consistent ordering;
generating a list of compressed representations of the plurality of transactions, wherein a compressed representation of the list is generated by computing a hash output based at least in part on a transaction of the plurality of transactions;
providing the list of compressed representations in place of the plurality of transactions;
receiving, at a receiver computer system, the list of compressed representations;
generating a second list of compressed representations from a mempool;
selecting a compressed representation of the received list of compressed representations; and
resolving the selected compressed representation against the second list based at least in part on whether:
the selected compressed representation is not found in the second list;
the selected compressed representation is found in the second list exactly once; or
the selected compressed representation is found in the second list multiple times.
2. The method of claim 1 , wherein as a result of the selected compressed representation being found in the second list multiple times:
performing a second-stage resolution by at least iterating through the multiple matches to determine whether any of the multiple matches resolve to a Merkle root of the block.
3. The method of claim 1 , wherein the method is implemented on a distributed system.
4. The method of claim 1 , wherein:
the hash output is computed further based at least in part on a cryptographic salt that is associated with the block of the blockchain; and
a different block of the blockchain network is associated with a different cryptographic salt.
5. The method of claim 1 , wherein the list of compressed representations is a list of transaction identifiers corresponding to the plurality of transactions, wherein the plurality of transactions are blockchain transactions.
6. The method of claim 1 , wherein providing the list of compressed representations in place of the plurality of transactions yields at least a 10× bandwidth savings.
7. A system comprising one or more processors and memory that stores computer-readable instructions that, as a result of execution by the one or more processors, cause the system to:
sort a plurality of transactions of a block of a blockchain network based at least in part on a consistent ordering;
generate a list of compressed representations of the plurality of transactions, wherein a compressed representation of the list is generated by computing a hash output based at least in part on a transaction of the plurality of transactions;
provide the list of compressed representations in place of the plurality of transactions; and
wherein the system comprises a receiver computer system, wherein the receiver computer system comprises second one or more processors and stores second computer-readable instructions that, as a result of execution by the second one or more processors, causes the receiver computer system to:
receive the list of compressed representations;
generate a second list of compressed representations from a mempool;
select a compressed representation of the received list of compressed representations; and
resolve the selected compressed representation against the second list based at least in part on whether:
the selected compressed representation is not found in the second list;
the selected compressed representation is found in the second list exactly once; or
the selected compressed representation is found in the second list multiple times.
8. The system of claim 7 , wherein the second computer-readable instructions include further instructions that, as a result of execution by the second one or more processors, further causes the receiver computer system to:
as a result of the selected compressed representation being found in the second list multiple times, perform a second-stage resolution by at least iterating through the multiple matches to determine whether any of the multiple matches resolve to a Merkle root of the block.
9. The system of claim 7 , wherein the system is a full node of the blockchain network.
10. The system of claim 7 , wherein:
the instructions to generate the list of compressed representations of the plurality of transactions include instructions to compute the hash output based at least in part on a cryptographic salt that is associated with the block of the blockchain; and
a different block of the blockchain network is associated with a different cryptographic salt.
11. The system of claim 7 , wherein the list of compressed representations is a list of transaction identifiers corresponding to the plurality of transactions, wherein the plurality of transactions are blockchain transactions.
12. The system of claim 7 , wherein the instructions to provide the list of compressed representations in place of the plurality of transactions, as a result of execution by the one or more processors, yields at least a 100× bandwidth savings.
13. A non-transitory computer-readable medium storing executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to:
sort a plurality of transactions of a block of a blockchain network based at least in part on a consistent ordering;
generate a list of compressed representations of the plurality of transactions, wherein a compressed representation of the list is generated by computing a hash output based at least in part on a transaction of the plurality of transactions;
provide the list of compressed representations in place of the plurality of transactions; and
wherein the executable instructions include further instructions that, as a result of being executed by second one or more processors of a second computer system, causes the second computer system to:
receive the list of compressed representations;
generate a second list of compressed representations from a mempool;
select a compressed representation of the received list of compressed representations; and
resolve the selected compressed representation against the second list based at least in part on whether:
the selected compressed representation is not found in the second list;
the selected compressed representation is found in the second list exactly once; or
the selected compressed representation is found in the second list multiple times.
14. The non-transitory computer-readable medium of claim 13 , wherein the instructions include further instructions that, as a result of execution by the second one or more processors, further causes the second computer system to:
as a result of the selected compressed representation being found in the second list multiple times, perform a second-stage resolution by at least iterating through the multiple matches to determine whether any of the multiple matches resolve to a Merkle root of the block.
15. The non-transitory computer-readable medium of claim 13 , wherein the computer system is or includes a distributed computer system.
16. The non-transitory computer-readable medium of claim 13 , wherein:
the instructions to generate the list of compressed representations of the plurality of transactions include instructions to compute the hash output based at least in part on a cryptographic salt that is associated with the block of the blockchain; and
a different block of the blockchain network is associated with a different cryptographic salt.
17. The non-transitory computer-readable medium of claim 13 , wherein the list of compressed representations is a list of transaction identifiers corresponding to the plurality of transactions, wherein the plurality of transactions are blockchain transactions.