Log-structured host data storage
View Patent ↗Methods of communicating data between a host and a client. The methods may comprise the step of receiving a write request from a client. The write request may comprise first data and first metadata describing a logical position of the first data within a file. The first data may be encrypted to the host. The methods may also comprise the step of writing the first data to a data storage. The first data may be written as a data sub-file of the file. In addition, the methods may comprise the step of writing the first metadata to the data storage. The first metadata may be written as a metadata sub-file of the file.
1. A method of communicating data between a host computer and a client computer, the method comprising:
receiving, by the host computer, a write request from the client computer, the write request comprising first data and first metadata describing a first logical position of the first data within a file, wherein the first data is encrypted to the host computer, wherein the first metadata is unencrypted to the host computer, wherein the host computer comprises a processor and operatively associated data storage, and wherein the file is stored at the data storage;
writing the first data to the data storage, wherein the first data is written as a data sub-file of the file, wherein the data sub-file logically replaces a second data sub-file having the same logical position;
creating a metadata sub-file utilizing the first metadata, wherein the metadata sub-file comprises logical-to-physical mapping information linking the first logical position of the first data within the file to the physical position of the data sub-file within the file; and
writing the metadata sub-file to the data storage, wherein the data sub-file and the metadata sub-file are organized within the file according to a log-structured organization.
2. The method of claim 1 , wherein the file comprises a plurality of logical positions, and wherein first logical position is selected from the plurality of logical positions.
3. The method of claim 2 , wherein the data sub-file is logically positioned after a plurality of data sub-files.
4. The method of claim 1 , wherein the first data and the first metadata are encrypted to parties other than the client and the host.
5. The method of claim 1 , wherein the first data is compressed according to a compression algorithm.
6. The method of claim 1 , further comprising:
receiving a read request from the client, wherein the read request identifies a second logical position within the file;
identifying a second data sub-file organized within the file according to the log-structured organization system;
determining a position of the second data sub-file within the file considering at least one metadata sub-file; and
communicating the second data sub-file to the client.
7. The method of claim 6 , further comprising:
generating a second metadata sub-file comprising a mapping of the second logical position within the file to a physical position of the second data sub-file within the file; and
communicating the second metadata to the client.
8. A system for communicating data between a client and a host, the system comprising:
a host comprising:
data storage hardware comprising thereon a plurality of files organized on the data storage hardware, and wherein each of the plurality of files comprises a plurality of sub-files organized within the file according to a log-structured organization, wherein the plurality of sub-files comprise at least one data sub-file and at least one metadata sub-file, wherein the at least one data sub-file is encrypted to the host and wherein the at least one metadata sub-file is unencrypted to the host; and
at least one server in communication with the data storage hardware and configured to:
receive a write request from a client, the write request comprising first data and first metadata describing a first logical position of the first data within a file, wherein the first data is encrypted to the host, and wherein the first metadata is unencrypted to the host;
write the first data to the data storage hardware, wherein the first data is written as a data sub-file of the file organized within the file according to the log-structured organization, wherein the data sub-file logically replaces a second data sub-file having the same Logical position;
create a metadata sub-file utilizing the first metadata, wherein the metadata sub-file comprises logical-to-physical mapping information linking the first logical position of the first data within the file to the physical position of the data sub-file within the file; and
write the metadata sub-file to the data storage hardware, wherein the data sub-file and the metadata sub-file are organized within the file according to a log-structured organization.
9. The system of claim 8 , wherein the at least one server is further configured to:
receive a read request from the client, wherein the read request identifies a second logical position within the logical unit;
identify a second data sub-file organized within the file according to the log-structured organization system; and
communicate the second data sub-file to the client.
10. The system of claim 9 , wherein the server is further configured to:
generate a second metadata sub-file comprising a mapping of the second logical position within the file to a physical position of the second data sub-file within the file; and
communicate the second metadata to the client.