System and method for data storage, transfer, synchronization, and security using recursive encoding
A system and method for data storage, transfer, synchronization, and security using recursive encoding, wherein data is deconstructed into chunklets, and is processed through a series of reference code libraries that reduce the data to a sequence of reference codes, and where the output of each reference library is used as the input to the next.
1. A system for storing, retrieving, and transmitting data in a highly compact format, comprising:
a computing device comprising a processor, a memory, and a first plurality of programming instructions;
a plurality of reference code libraries stored in the memory of the computing device, wherein a first reference code library comprises data chunklets and reference codes to the data chunklets, and each subsequent reference code library comprises groups of reference codes from the previous reference code library and new reference codes to those groups of references codes from the previous reference code library; and
a recursive deconstruction engine comprising a second plurality of programming instructions stored in the memory and operable on the processor, wherein the programming instructions, when operating on the processor, cause the processor to:
receive data;
deconstruct the data into a plurality of data chunklets;
encode the data using the first reference code library by:
retrieving the reference code for each chunklet from the first reference code library; and
where there is no reference code for a given chunklet, creating a new reference code, and storing the chunklet and its newly-created reference code in the first reference code library; and
recursively encode the data using each subsequent reference code library by:
retrieving the reference code in a current reference code library for each group of reference codes corresponding to the data from the previous library; and
where there is no reference code for a given group of reference codes corresponding to the data in the current reference code library, creating a new reference code, and storing the group of reference codes from the previous reference code library and its newly-created reference code in the current reference code library; and
storing the reference codes corresponding to the data from the last subsequent reference code library used to encode the data; and
a recursive reconstruction engine comprising a plurality of programming instructions stored in the memory and operable on the processor, wherein the programming instructions, when operating on the processor, cause the processor to:
retrieve the stored reference codes corresponding to the data;
recursively decode the data using each previous reference code library by:
retrieving, from the current code library, the group of reference codes for each reference code corresponding to the data in the current code library;
using the groups of reference codes from the current reference code library to decode groups of reference codes using the previous reference code library; and
repeating the decoding process at each previous reference code library until the chunklets corresponding to the data are decoded using the first reference code library; and
assembling the chunklets to reconstruct the data.
2. The system of claim 1 , wherein the data to be recursively encoded comprises one or more reference code libraries.
3. A method for storing, retrieving, and transmitting data in a highly compact format, comprising the steps of:
storing a plurality of reference code libraries, wherein a first reference code library comprises data chunklets and reference codes to the data chunklets, and each subsequent reference code library comprises groups of reference codes from the previous reference code library and new reference codes to those groups of references codes from the previous reference code library; and
receiving data;
deconstructing the data into a plurality of data chunklets;
encoding the data using the first reference code library by:
retrieving the reference code for each chunklet from the first reference code library; and
where there is no reference code for a given chunklet, creating a new reference code, and storing the chunklet and its newly-created reference code in the first reference code library; and
recursively encoding the data using each subsequent reference code library by:
retrieving the reference code in a current reference code library for each group of reference codes corresponding to the data from the previous library; and
where there is no reference code for a given group of reference codes corresponding to the data in the current reference code library, creating a new reference code, and storing the group of reference codes from the previous reference code library and its newly-created reference code in the current reference code library; and
storing the reference codes corresponding to the data from the last subsequent reference code library used to encode the data; and
retrieving the stored reference codes corresponding to the data;
recursively decoding the data using each previous reference code library by:
retrieving, from the current code library, the group of reference codes for each reference code corresponding to the data in the current code library;
using the groups of reference codes from the current reference code library to decode groups of reference codes using the previous reference code library; and
repeating the decoding process at each previous reference code library until the chunklets corresponding to the data are decoded using the first reference code library; and
assembling the chunklets to reconstruct the data.
4. The method of claim 3 , wherein the data to be recursively encoded comprises one or more reference code libraries.