IP Library Granted Patent US 8,762,343
Granted Patent B2
US 8,762,343 · App. 12/902,684 · Granted Jun 24, 2014

Dispersed storage of software

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,762,343
App. No.
12/902,684
Granted
Jun 24, 2014
Kind
B2
Abstract

A data de-duplication method begins by a processing module receiving a plurality of data storage requests from a plurality of requesting devices wherein a data storage request includes the data and a requester identifier (ID). The method continues with the processing module obtaining a data identifier (ID) for the data. For each of the plurality of data storage requests, the method continues with the processing module producing a requester storage record, dispersed storage error encoding the requester storage record to produce a set of encoded requester storage record slices, and sending the set of encoded requester storage record slices to a dispersed storage network (DSN) memory for storage therein. The method continues with the processing module dispersed storage error encoding at least a portion of the data to produce a set of encoded data slices and sending the set of encoded data slices to the DSN memory for storage therein.

Claims (122)

1. A data de-duplication method, wherein the method comprises:

receiving a plurality of data storage requests from a plurality of requesting devices regarding storage of substantially identical data, wherein a data storage request of the plurality of data storage requests includes the substantially identical data and a requester identifier (ID) of a corresponding one of the plurality of requesting devices;

obtaining a data identifier (ID) for the substantially identical data;

generating a requester storage record to include the data ID and the requester IDs from each of the plurality of data storage requests;

dispersed storage error encoding the requester storage record to produce a set of encoded requester storage record slices;

sending the set of encoded requester storage record slices to a dispersed storage network (DSN) memory for storage therein;

for the plurality of data storage requests, dispersed storage error encoding at least a portion of the substantially identical data from one of the plurality of data storage requests to produce a set of encoded data slices; and

sending the set of encoded data slices to the DSN memory for storage therein in accordance with the data ID.

2. The method of claim 1 , wherein the obtaining the data ID comprises at least one of:

generating a calculated hash of the substantially identical data;

retrieving the data ID from profile table based on the calculated hash; and

receiving the data ID from the requester.

3. The method of claim 1 further comprises:

determining, from data storage request to data storage request of the plurality of data storage requests, whether the substantially identical data is substantially the same by at least one of:

determining whether a first data ID associated with the substantially identical data of a first data storage request of the plurality of data storage requests substantially matches a second data ID associated with the substantially identical data of a second data storage request of the plurality of data storage requests;

determining whether a first hash of the substantially identical data of the first data storage request substantially matches a second hash of the substantially identical data of the second data storage request; and

comparing the substantially identical data of the first data storage request with the substantially identical data of the second data storage request.

4. The method of claim 1 , wherein the combining the data ID and the requester ID comprises at least one of:

selecting at least one of the data ID and the requester ID as the requester storage record;

encrypting at least one of the data ID and the requester ID using a random encryption key to produce the requester storage record;

encrypting the random encryption key using a public key to produce an encrypted random encryption key;

dispersed storage error encoding the encrypted random encryption key to produce a set of encoded key slices; and

sending the set of encoded key slices to the DSN memory for storage therein.

5. The method of claim 1 , wherein the dispersed storage error encoding at least the portion of the substantially identical data comprises at least one of:

encoding at least one of the portion of the substantially identical data and the data ID using an error coding dispersal storage function to produce the set of encoded data slices;

encrypting at least one of the portion of the substantially identical data and the data ID using a random encryption key to produce encrypted data and encoding the encrypted data using the error coding dispersal storage function to produce the set of encoded data slices; and

encrypting the random encryption key using a public key to produce an encrypted random encryption key and dispersed storage error encoding the encrypted random encryption key to produce a set of encoded key slices.

6. The method of claim 1 further comprises:

selecting the one of the plurality of data storage requests based on at least one of:

an ordering of receipt of the plurality of data storage requests;

the requester ID identifying a particular de-duplication authorization level;

a type of the data;

a size of the data; and

number of the plurality of data storage requests.

7. A method for retrieving de-duplicated stored data, wherein the method comprises:

receiving a plurality of data retrieval requests from a plurality of requesting devices to retrieve substantially identical data, wherein a data retrieval request of the plurality of data retrieval requests includes a requester storage record identifier (ID);

for at least one of the plurality of data retrieval requests:

retrieving a set of encoded requester storage record slices from a dispersed storage network (DSN) memory based on the requester storage record ID;

dispersed storage error decoding the set of encoded requester storage record slices to reproduce a requester storage record; and

extracting a data identifier (ID) and a requester identifier (ID) from the requester storage record, wherein the data ID identifies the substantially identical data;

retrieving a set of encoded data slices from the DSN memory based on the data ID of one of the plurality of data retrieval requests;

dispersed storage error decoding the set of encoded data slices to reproduce at least a portion of the de-duplicated stored data; and

sending the at least a portion of the de-duplicated stored data to each of the plurality of requesting devices in accordance with the plurality of extracted requester IDs.

8. The method of claim 7 further comprises:

the data retrieval request further includes a requester device provided requester ID; and

verifying the requester ID extracted from the requester storage record with the requester device provided requester ID.

9. The method of claim 7 , wherein the dispersed storage error decoding the set of encoded requester storage record slices comprises:

retrieving a set of encoded key slices from the DSN memory;

dispersed storage error decoding the set of encoded key slices to produce an encrypted random encryption key;

decrypting the encrypted random encryption key using a private key to produce a random encryption key;

dispersed storage error decoding the set of encoded requester storage record to produce an encrypted requester storage record; and

decrypting the encrypted requester storage record using the random encryption key to produce the requester storage record.

10. The method of claim 7 , wherein the dispersed storage error decoding the set of encoded data slices comprises at least one of:

retrieving a set of encoded key slices from the DSN memory;

dispersed storage error decoding the set of encoded key slices to produce an encrypted random encryption key;

decrypting the encrypted random encryption key using a private key to produce a random encryption key;

dispersed storage error decoding the plurality of encoded data slices to produce a plurality of encrypted data segments; and

decrypting each of the plurality of encrypted data segments using the random encryption key to produce the at least a portion of the de-duplicated stored data.

11. A computer comprises:

an interface;

memory: and

one or more processing devices operably coupled to the interface and the memory, wherein the one or more processing devices is operable to:

receive, via the interface, a plurality of data storage requests from a plurality of requesting devices regarding storage of substantially identical data, wherein a data storage request of the plurality of data storage requests includes the substantially identical data and a requester identifier (ID) of a corresponding one of the plurality of requesting devices;

obtain a data identifier (ID) for the substantially identical data;

generate a requester storage record to include the data ID and the requester IDs from each of the plurality of data storage requests;

dispersed storage error encode the requester storage record to produce a set of encoded requester storage record slices;

send, via the interface, the set of encoded requester storage record slices to a dispersed storage network (DSN) memory for storage therein;

for the plurality of data storage requests, dispersed storage error encode at least a portion of the substantially identical data from one of the plurality of data storage requests to produce a set of encoded data slices; and

send, via interface, the set of encoded data slices to the DSN memory for storage therein in accordance with the data ID.

12. The computer of claim 11 , wherein the one or more processing devices further functions to obtain the data ID by least one of:

generating a calculated hash of the substantially identical data;

retrieving the data ID from profile table based on the calculated hash; and

receiving, via the interface, the data ID from the requester.

13. The computer of claim 11 , wherein the one or more processing devices further functions to:

determine, from data storage request to data storage request of the plurality of data storage requests, whether the substantially identical data is substantially the same by at least one of:

determining whether a first data ID associated with the substantially identical data of a first data storage request of the plurality of data storage requests substantially matches a second data ID associated with the substantially identical data of a second data storage request of the plurality of data storage requests;

determining whether a first hash of the substantially identical data of the first data storage request substantially matches a second hash of the substantially identical data of the second data storage request; and

comparing the substantially identical data of the first data storage request with the substantially identical data of the second data storage request.

14. The computer of claim 11 , wherein the one or more processing devices further functions to combine the data ID and the requester ID by at least one of:

selecting at least one of the data ID and the requester ID as the requester storage record;

encrypting at least one of the data ID and the requester ID using a random encryption key to produce the requester storage record;

encrypting the random encryption key using a public key to produce an encrypted random encryption key;

dispersed storage error encoding the encrypted random encryption key to produce a set of encoded key slices; and

sending, via the interface, the set of encoded key slices to the DSN memory for storage therein.

15. The computer of claim 11 , wherein the one or more processing devices further functions to dispersed storage error encode at least the portion of the substantially identical data by at least one of:

encoding at least one of the portion of the substantially identical data and the data ID using an error coding dispersal storage function to produce the set of encoded data slices;

encrypting at least one of the portion of the substantially identical data and the data ID using a random encryption key to produce encrypted data and encoding the encrypted data using the error coding dispersal storage function to produce the set of encoded data slices; and

encrypting the random encryption key using a public key to produce an encrypted random encryption key and dispersed storage error encoding the encrypted random encryption key to produce a set of encoded key slices.

16. The computer of claim 11 , wherein the one or more processing devices further functions to:

select the one of the plurality of data storage requests based on at least one of:

an ordering of receipt of the plurality of data storage requests;

the requester ID identifying a particular de-duplication authorization level;

a type of the data;

a size of the data; and

number of the plurality of data storage requests.

17. A computer comprises:

an interface;

memory: and

one or more processing devices operably coupled to the interface and the memory, wherein the one or more processing devices is operable to:

receive, via the interface, a plurality of data retrieval requests from a plurality of requesting devices to retrieve substantially identical data, wherein a data retrieval request of the plurality of data retrieval requests includes a requester storage record identifier (ID);

for at least one of the plurality of data retrieval requests:

retrieve, via the interface, a set of encoded requester storage record slices from a dispersed storage network (DSN) memory based on the requester storage record ID;

dispersed storage error decode the set of encoded requester storage record slices to reproduce a requester storage record; and

extract a data identifier (ID) and a requester identifier (ID) from the requester storage record, wherein the data ID identifies the substantially identical data;

retrieve, via the interface, a set of encoded data slices from the DSN memory based on the data ID of one of the plurality of data retrieval requests;

dispersed storage error decode the set of encoded data slices to reproduce at least a portion of the de-duplicated stored data; and

send, via the interface, the at least a portion of the de-duplicated stored data to each of the plurality of requesting devices in accordance with the plurality of extracted requester IDs.

18. The computer of claim 17 , wherein the one or more processing devices further functions to:

the data retrieval request further includes a requester device provided requester ID; and

verify the requester ID extracted from the requester storage record with the requester device provided requester ID.

19. The computer of claim 17 , wherein the one or more processing devices functions to dispersed storage error decode the set of encoded requester storage record slices by:

retrieving, via the interface, a set of encoded key slices from the DSN memory;

dispersed storage error decoding the set of encoded key slices to produce an encrypted random encryption key;

decrypting the encrypted random encryption key using a private key to produce a random encryption key;

dispersed storage error decoding the set of encoded requester storage record to produce an encrypted requester storage record; and

decrypting the encrypted requester storage record using the random encryption key to produce the requester storage record.

20. The computer of claim 17 , wherein the one or more processing devices functions to dispersed storage error decode the set of encoded data slices by at least one of:

retrieving, via the interface, a set of encoded key slices from the DSN memory;

dispersed storage error decoding the set of encoded key slices to produce an encrypted random encryption key;

decrypting the encrypted random encryption key using a private key to produce a random encryption key;

dispersed storage error decoding the plurality of encoded data slices to produce a plurality of encrypted data segments; and

decrypting each of the plurality of encrypted data segments using the random encryption key to produce the at least a portion of the de-duplicated stored data.

Assignments (6)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jun 11, 2025
From: BARCLAYS BANK PLC, AS ADMINISTRATIVE AGENT
To: PURE STORAGE, INC.
Reel/Frame 071558/0523 →
SECURITY INTEREST Recorded Aug 26, 2020
From: PURE STORAGE, INC.
To: BARCLAYS BANK PLC AS ADMINISTRATIVE AGENT
Reel/Frame 053867/0581 →
CORRECTIVE ASSIGNMENT TO CORRECT THE 9992063 AND 10334045 LISTED IN ERROR PREVIOUSLY RECORDED ON REEL 049556 FRAME 0012. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNOR HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 14, 2020
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: PURE STORAGE, INC.
Reel/Frame 052205/0705 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: PURE STORAGE, INC.
Reel/Frame 049556/0012 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2016
From: CLEVERSAFE, INC.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 038687/0596 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2010
From: GRUBE, GARY W.; MARKISON, TIMOTHY W.
To: CLEVERSAFE, INC.
Reel/Frame 025125/0605 →