DEDUPLICATED DATA DISTRIBUTION TECHNIQUES
In connection with a data distribution architecture, client-side “deduplication” techniques may be utilized for data transfers occurring among various file system nodes. In some examples, these deduplication techniques involve fingerprinting file system elements that are being shared and transferred, and dividing each file into separate units referred to as “blocks” or “chunks.” These separate units may be used for independently rebuilding a file from local and remote collections, storage locations, or sources. The deduplication techniques may be applied to data transfers to prevent unnecessary data transfers, and to reduce the amount of bandwidth, processing power, and memory used to synchronize and transfer data among the file system nodes. The described deduplication concepts may also be applied for purposes of efficient file replication, data transfers, and file system events occurring within and among networks and file system nodes.
1 . A method for a deduplication-based reconstruction of file system data, the method comprising operations performed by at least one processor of a first computing system, and the operations including:
transmitting, to a second computing system, a request for metadata of a desired file;
receiving, from the second computing system, the metadata of the desired file, the metadata of the desired file indicating respective identifiers of each block of the desired file;
identifying, with use of the metadata of the desired file, at least one block of the desired file on a data store associated with the first computing system; and
reconstructing the desired file with use of the at least one block of the desired file on the data store, and with use of the metadata of the desired file received from the second computing system.