IP Library Granted Patent US 9,015,556
Granted Patent B2
US 9,015,556 · App. 13/707,542 · Granted Apr 21, 2015

Transforming data in a distributed storage and task network

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,015,556
App. No.
13/707,542
Granted
Apr 21, 2015
Kind
B2
Abstract

A method begins by a dispersed storage (DS) processing module determining whether at least a portion of temporarily stored data is to be stored long-term, wherein the temporarily stored data is stored in a set of distributed storage and task (DST) units in accordance with a computational-orientated dispersed storage error coding function. When the at least a portion of the temporarily stored data is to be stored long-term, the method continues with the DS processing module identifying one or more DST storing the at least a portion of the temporarily stored data, recovering the at least a portion of the temporarily stored data, dispersed storage error encoding the at least a portion of the temporarily stored data in a pre-dispersed storage error encoded format into a plurality of sets of encoded data slices, and storing the plurality of sets of encoded data slices in the set of DST units.

Claims (99)

1. A method for execution by a device within a distributed storage and task network, the method comprises:

determining whether at least a portion of temporarily stored data is to be stored long-term, wherein the temporarily stored data is stored in a set of distributed storage and task (DST) units in accordance with a computational-orientated dispersed storage error coding function;

when the at least a portion of the temporarily stored data is to be stored long-term:

identifying one or more DST units of the set of DST units storing the at least a portion of the temporarily stored data in accordance with the computational-orientated dispersed storage error coding function;

recovering the at least a portion of the temporarily stored data from the one or more DST units in a pre-dispersed storage error encoded format to produce recovered data;

dispersed storage error encoding the recovered data into a plurality of sets of encoded data slices in accordance with a long-term-storage-orientated dispersed storage error coding function; and

storing the plurality of sets of encoded data slices in the set of DST units.

2. The method of claim 1 further comprises:

the computational-orientated dispersed storage error coding function including first dispersed storage error coding parameters that enables a DST unit of the set of DST units to recover, in the pre-dispersed storage error encoded format, a sub-portion of the portion of the temporarily stored data from encoded data slices the DST unit stores; and

the long-term-storage-orientated dispersed storage error coding function including second dispersed storage error coding parameters that prevents the DST unit from recovering, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data from the encoded data slices the DST unit stores and requires retrieval of encoded data slices from multiple DST units of the set of DST units to recover, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data.

3. The method of claim 1 further comprises:

temporarily storing raw data, as the temporarily stored data, in the set of DST units in accordance with the computational-orientated dispersed storage error coding function;

performing, by at least some of the DST units, a task on the temporarily stored data to produce found data; and

dispersed storage error encoding the found data, as the at least a portion of the temporarily stored data, into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

4. The method of claim 1 further comprises:

temporarily storing found data, as the temporarily stored data, in the set of DST units in accordance with the computational-orientated dispersed storage error coding function, wherein at least some of the DST units performed a task on raw data to produce the found data;

performing, by the at least some of the DST units, another task on the temporarily stored data to produce a sub-set of found data; and

dispersed storage error encoding the sub-set of found data, as the at least a portion of the temporarily stored data, into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

5. The method of claim 1 , wherein the dispersed storage error encoding the at least a portion of the temporarily stored data comprises:

determining, in accordance with the long-term-storage-orientated dispersed storage error coding function, a data mapping of data-based encoded data slices of the at least a portion of the temporarily stored data in a pre-dispersed storage error encoded format;

generating redundancy-based encoded data slices based on the data-based encoded data slices and in accordance with the long-term-storage-orientated dispersed storage error coding function; and

organizing the data-based encoded data slices and the redundancy-based encoded data slices into the plurality of sets of encoded data slices.

6. The method of claim 1 , wherein the dispersed storage error encoding the at least a portion of the temporarily stored data comprises:

dispersed storage error encoding the at least a portion of the temporarily stored data in the pre-dispersed storage error encoded format into a plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function; and

organizing the plurality of encoded data blocks into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

7. The method of claim 1 , wherein the identifying the one or more DST units of the set of DST units comprises at least one of:

receiving a message regarding the at least a portion of the temporarily stored data, wherein the message identifies the one or more DST units;

receiving the message regarding the at least a portion of the temporarily stored data and determining the one or more DST units based storage of the at least a portion of the temporarily stored data; and

determining that no further computations are to be performed on the at least a portion of the temporarily stored data.

8. A method for execution by a device within a distributed storage and task network, the method comprises:

determining that a task is to be performed on data, wherein the data is encoded into a plurality of encoded data blocks in accordance with a long-term-storage-orientated dispersed storage error coding function and the plurality of encoded data blocks are stored in a set of distributed storage and task (DST) units;

requesting reconfiguration of storage of the data from the long-term-storage-orientated dispersed storage error coding function to a computational-orientated dispersed storage error coding function, wherein the data is encoded into groupings of encoded data blocks in accordance with the computational-orientated dispersed storage error coding function;

requesting storage of the groupings of encoded data blocks by the set of DST units, wherein a DST unit of the set of DST units recovers a portion of the data from one of the groupings of encoded data blocks and performs a portion of the task on the recovered portion of the data to produce a partial task resultant;

receiving partial task resultants from the at least some of the set of DST units; and

compiling the partial task resultants to produce a task resultant.

9. The method of claim 8 further comprises:

the computational-orientated dispersed storage error coding function including first dispersed storage error coding parameters that enables the DST unit of the set of DST units to recover, in a pre-dispersed storage error encoded format, a sub-portion of the portion of the temporarily stored data from encoded data slices the DST unit stores; and

the long-term-storage-orientated dispersed storage error coding function including second dispersed storage error coding parameters that prevents the DST unit from recovering, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data from the encoded data slices the DST unit stores and requires retrieval of encoded data slices from multiple DST units of the set of DST units to recover, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data.

10. The method of claim 8 , wherein the reconfiguring storage of the data comprises:

decoding the plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function to recover the data;

encoding the recovered data in accordance with the computational-orientated dispersed storage error coding function to produce the groupings of encoded data blocks; and

sending one of the groupings of encoded data blocks to the DST unit.

11. The method of claim 8 , wherein the reconfiguring storage of the data comprises:

identifying data-based encoded data slices of the plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function;

determining a data mapping of the data-based encoded data slices between the set of DST units and the at least some of the DST units; and

facilitating copying of at least some of the data-based encoded data slices to the at least some of the set of DST units in accordance with the data mapping.

12. A dispersed storage (DS) module of within a distributed storage and task network comprises:

a first module, when operable within a computing device, causes the computing device to:

determine whether at least a portion of temporarily stored data is to be stored long-term, wherein the temporarily stored data is stored in a set of distributed storage and task (DST) units in accordance with a computational-orientated dispersed storage error coding function; and

a second module, when operable within the computing device, causes the computing device to:

when the at least a portion of the temporarily stored data is to be stored long-term:

identify one or more DST units of the set of DST units storing the at least a portion of the temporarily stored data in accordance with the computational-orientated dispersed storage error coding function;

recover the at least a portion of the temporarily stored data from the one or more DST units in a pre-dispersed storage error encoded format to produce recovered data;

dispersed storage error encode the recovered data into a plurality of sets of encoded data slices in accordance with a long-term-storage-orientated dispersed storage error coding function; and

store the plurality of sets of encoded data slices in the set of DST units.

13. The DS module of claim 12 further comprises:

the computational-orientated dispersed storage error coding function including first dispersed storage error coding parameters that enables a DST unit of the set of DST units to recover, in the pre-dispersed storage error encoded format, a sub-portion of the portion of the temporarily stored data from encoded data slices the DST unit stores; and

the long-term-storage-orientated dispersed storage error coding function including second dispersed storage error coding parameters that prevents the DST unit from recovering, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data from the encoded data slices the DST unit stores and requires retrieval of encoded data slices from multiple DST units of the set of DST units to recover, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data.

14. The DS module of claim 12 further comprises:

a third module, when operable within the computing device, causes the computing device to:

temporarily store raw data, as the temporarily stored data, in the set of DST units in accordance with the computational-orientated dispersed storage error coding function;

perform, by at least some of the DST units, a task on the temporarily stored data to produce found data; and

dispersed storage error encode the found data, as the at least a portion of the temporarily stored data, into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

15. The DS module of claim 12 further comprises:

a fourth module, when operable within the computing device, causes the computing device to:

temporarily store found data, as the temporarily stored data, in the set of DST units in accordance with the computational-orientated dispersed storage error coding function, wherein at least some of the DST units performed a task on raw data to produce the found data;

perform, by the at least some of the DST units, another task on the temporarily stored data to produce a sub-set of found data; and

dispersed storage error encode the sub-set of found data, as the at least a portion of the temporarily stored data, into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

16. The DS module of claim 12 , wherein the second module functions to dispersed storage error encode the at least a portion of the temporarily stored data by:

determining, in accordance with the long-term-storage-orientated dispersed storage error coding function, a data mapping of data-based encoded data slices of the at least a portion of the temporarily stored data in a pre-dispersed storage error encoded format;

generating redundancy-based encoded data slices based on the data-based encoded data slices and in accordance with the long-term-storage-orientated dispersed storage error coding function; and

organizing the data-based encoded data slices and the redundancy-based encoded data slices into the plurality of sets of encoded data slices.

17. The DS module of claim 12 , wherein the second module functions to dispersed storage error encode the at least a portion of the temporarily stored data by:

dispersed storage error encoding the at least a portion of the temporarily stored data in the pre-dispersed storage error encoded format into a plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function; and

organizing the plurality of encoded data blocks into the plurality of sets of encoded data slices in accordance with the long-term-storage-orientated dispersed storage error coding function.

18. The DS module of claim 12 , wherein the second module functions to identify one or more DST units of the set of DST units by at least one of:

receiving a message regarding the at least a portion of the temporarily stored data, wherein the message identifies the one or more DST units;

receiving the message regarding the at least a portion of the temporarily stored data and determining the one or more DST units based storage of the at least a portion of the temporarily stored data; and

determining that no further computations are to be performed on the at least a portion of the temporarily stored data.

19. A dispersed storage (DS) module of within a distributed storage and task network comprises:

a first module, when operable within a computing device, causes the computing device to:

determine that a task is to be performed on data, wherein the data is encoded into a plurality of encoded data blocks in accordance with a long-term-storage-orientated dispersed storage error coding function and the plurality of encoded data blocks are stored in a set of distributed storage and task (DST) units;

a second module, when operable within the computing device, causes the computing device to:

request reconfiguration of storage of the data from the long-term-storage-orientated dispersed storage error coding function to a computational-orientated dispersed storage error coding function, wherein the data is encoded into groupings of encoded data blocks in accordance with the computational-orientated dispersed storage error coding function; and

a third module, when operable within the computing device, causes the computing device to:

request storage of the groupings of encoded data blocks by the set of DST units, wherein a DST unit of the set of DST units recovers a portion of the data from one of the groupings of encoded data blocks and performs a portion of the task on the recovered portion of the data to produce a partial task resultant; and

receive partial task resultants from the at least some of the set of DST units; and

compile the partial task resultants to produce a task resultant.

20. The DS module of claim 19 further comprises:

the computational-orientated dispersed storage error coding function including first dispersed storage error coding parameters that enables the DST unit of the set of DST units to recover, in a pre-dispersed storage error encoded format, a sub-portion of the portion of the temporarily stored data from encoded data slices the DST unit stores; and

the long-term-storage-orientated dispersed storage error coding function including second dispersed storage error coding parameters that prevents the DST unit from recovering, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data from the encoded data slices the DST unit stores and requires retrieval of encoded data slices from multiple DST units of the set of DST units to recover, in the pre-dispersed storage error encoded format, the sub-portion of the portion of the temporarily stored data.

21. The DS module of claim 19 , wherein the second module functions to reconfigure storage of the data by:

decoding the plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function to recover the data;

encoding the recovered data in accordance with the computational-orientated dispersed storage error coding function to produce the groupings of encoded data blocks; and

sending one of the groupings of encoded data blocks to the DST unit.

22. The DS module of claim 19 , wherein the second module functions to reconfigure storage of the data by:

identifying data-based encoded data slices of the plurality of encoded data blocks in accordance with the long-term-storage-orientated dispersed storage error coding function;

determining a data mapping of the data-based encoded data slices between the set of DST units and the at least some of the DST units; and

facilitating copying of at least some of the data-based encoded data slices to the at least some of the set of DST units in accordance with the data mapping.

Assignments (6)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jun 11, 2025
From: BARCLAYS BANK PLC, AS ADMINISTRATIVE AGENT
To: PURE STORAGE, INC.
Reel/Frame 071558/0523 →
SECURITY INTEREST Recorded Aug 26, 2020
From: PURE STORAGE, INC.
To: BARCLAYS BANK PLC AS ADMINISTRATIVE AGENT
Reel/Frame 053867/0581 →
CORRECTIVE ASSIGNMENT TO CORRECT THE 9992063 AND 10334045 LISTED IN ERROR PREVIOUSLY RECORDED ON REEL 049556 FRAME 0012. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNOR HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 14, 2020
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: PURE STORAGE, INC.
Reel/Frame 052205/0705 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: PURE STORAGE, INC.
Reel/Frame 049556/0012 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2016
From: CLEVERSAFE, INC.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 038687/0596 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2012
From: LEGGETTE, WESLEY; BAPTIST, ANDREW; DHUSE, GREG; RESCH, JASON K.; VOLVOVSKI, ILYA; MOTWANI, MANISH; GLADWIN, S. CHRISTOPHER; GRUBE, GARY W.; SHIRLEY, THOMAS FRANKLIN, JR.
To: CLEVERSAFE, INC.
Reel/Frame 029422/0551 →