Log access method storage control apparatus, archive system, and method of operation
View Patent ↗A file shard store includes high performance encoding and compaction circuits. An apparatus and its method of operation avoids duplicate storage of file fragments. A plurality of tables control write operations into a plurality of log segments. Shard keys are transferred to uniquely identify fragments of files which may have been previously stored and associated with one or more of client subscribers. An apparatus comprises a plurality of location/subscription tables, a shard transfer circuit, a storage control circuit, an append-to-log circuit coupled to a large storage array, and a log segment compactor circuit.
1. A method for operating a Log Access Method Storage Control Apparatus comprising:
receiving a command and an argument from a transmitting client,
when the command is Exists and the argument is a shard key,
reading a table for presence or absence of the shard key and returning the result to the transmitting client; when the command is Unmark, and the argument is a shard key,
removing the identity of the transmitting client from a log associated with a shard uniquely associated with the shard key,
when the command is Mark, and the argument is a shard key,
adding the identity of the transmitting client to a log associated with a shard uniquely associated with the shard key,
when the command is Get, and the argument is a shard key,
looking up a location of a shard uniquely associated with the shard key, and
transmitting the shard to the transmitting client, when the command is Put, and the arguments comprise a shard and a shard key,
storing the shard at the location,
storing the shard key, the location, and an identification of the transmitting client into a table, and
initializing a log file with at least the shard key and the identification of the transmitting client.
2. The method of claim 1 further comprising:
when the command is Get, and the argument is a shard key,
obtaining the location for each requested shard from a computer readable table, and
delivering the shard(s) to the client.
3. The method of claim 2 further comprising:
checking that the client is associated with the shard in the table.
4. The method of claim 1 further comprising:
sending the transmitting client at least one acknowledgement.
5. The method of claim 4 further comprising:
determining from a computer readable table that a shard uniquely associated with the shard key is already stored and associated with the transmitting client, wherein the acknowledgement is a success acknowledgement.
6. The method of claim 5 further comprising:
determining from a computer readable table that a shard uniquely associated with the shard key is already stored but not yet associated with the transmitting client,
appending to a log file associated with the shard, the additional identification of the transmitting client, and updating the table,
wherein the acknowledgement is a success acknowledgement.
7. The method of claim 6 further comprising:
when the command is Unmark, and the argument is a shard key,
determining from a table that a shard uniquely associated with the shard key is stored and associated with the transmitting client,
appending to a log file associated with the shard, a recission identification of the transmitting client, and updating the table,
wherein the acknowledgement is a success acknowledgement.
8. The method of claim 7 further comprising:
determining from a table that a shard uniquely associated with the shard key is not stored, wherein the first code is a shard transmission request code,
receiving the shard,
storing to a location,
initializing a log file associated with the shard, its location, and the identification of the transmitting client, and updating the table, and
transmitting a success acknowledgement to the client.
9. The method of claim 1 further comprising;
writing shard meta data, client subscription, and location data to a file associated with each shard location and client subscription table.
10. The method of claim 1 further comprising:
reading a file containing shard meta data, client subscription, and location data associated with a log segment.
11. The method of claim 1 further comprising:
receiving from a requestor a plurality of shard ids and client ids;
looking up a shard location for each shard on a condition that the client is a subscriber to each of the shard ids;
retrieving each requested shard; and
transmitting the retrieved shard to the requestor.