IP Library Granted Patent US 10,324,791
Granted Patent B2
US 10,324,791 · App. 15/362,460 · Granted Jun 18, 2019

Selectable parallel processing of dispersed storage error encoding

Inventors: Andrew D. Baptist (Mt. Pleasant, WI); Greg R. Dhuse (Chicago, IL); Jason K. Resch (Chicago, IL); Ilya Volvovski (Chicago, IL)
Assignee: International Business Machines Corporation
G06F11/1076G06F3/0619G06F3/0629G06F3/0652G06F3/0689G06F11/00G06F12/1408H04L67/02H04L67/1097H04L67/22H04L67/306G06F2212/7205
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,324,791
App. No.
15/362,460
Granted
Jun 18, 2019
Kind
B2
Abstract

A method includes a computing device determining dispersed storage error encoded parameters for dispersed storage error encoding a data object. The method further includes the computing device dividing the dispersed storage error encoding into a plurality of encoding tasks. The method further includes allocating, by the computing device, the encoding tasks to a plurality of central processing units (CPUs) available to the computing device. The method further includes dispersed storage error encoding, by the CPUs, the data segments into the plurality of sets of encoded data slices in accordance with the encoding tasks. The method further includes sending, by the computing device, the plurality of sets of encoded data slices to storage units of the DSN for storage therein.

Claims (47)

1. A method comprises:

determining, by a computing device of a dispersed storage network (DSN), dispersed storage error encoded parameters to be utilized in a dispersed storage error encoding process of a data object, wherein the dispersed storage error encoding process includes dividing the data object into a plurality of data segments and dispersed storage error encoding the plurality of the data segments into a plurality of sets of encoded data slices;

dividing, by the computing device, the dispersed storage error encoding process into a plurality of encoding tasks, wherein dividing the dispersed storage error encoding process into the plurality of encoding tasks includes:

creating a first encoding task of the plurality of encoding tasks, wherein the first encoding task is to perform a first matrix multiplication on a first portion of an encoded matrix and a first portion of a data matrix to produce one or more encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices; and

creating a second encoding task of the plurality of encoding tasks, wherein the second encoding task is to perform a second matrix multiplication on a second portion of the encoded matrix and a second portion of the data matrix to produce another one or more encoded data slices of the set of encoded data slices;

allocating, by the computing device, the plurality of encoding tasks to a plurality of central processing units (CPUs) available to the computing device;

dispersed storage error encoding, by the plurality of CPUs, the plurality of data segments into the plurality of sets of encoded data slices in accordance with the plurality of encoding tasks; and

sending, by the computing device, the plurality of sets of encoded data slices to storage units of the DSN for storage therein.

2. The method of claim 1 further comprises:

determining a number of central processing units available for the dispersed storage error encoding process of the data object; and

selecting the plurality of CPUs from the number of central processing units.

3. The method of claim 1 , wherein dividing the dispersed storage error encoding process into a plurality of encoding tasks comprises:

creating an encoding task of the plurality of encoding tasks to be the dividing of the data object into the plurality of data segments.

4. The method of claim 1 , wherein dividing the dispersed storage error encoding process into a plurality of encoding tasks comprises:

creating an encoding task of the plurality of encoding tasks to generate the data matrix for a data segment of the plurality of data segments.

5. The method of claim 1 , wherein dividing the dispersed storage error encoding process into a plurality of encoding tasks comprises:

creating a first intermediate encoding task of the first encoding task to perform a first intermediate matrix multiplication on a first sub-portion of the first portion of the encoded matrix and a first sub-portion of the first portion of the data matrix to produce one or more encoded data slice portions of an encoded data slice of the one or more encoded data slices; and

creating a second intermediate encoding task of the first encoding task to perform a second intermediate matrix multiplication on a second sub-portion of the first portion of the encoded matrix and a second sub-portion of the first portion of the data matrix to produce another one or more encoded data slice portions of the encoded data slice; and

creating a third intermediate encoding task of the first encoding task to aggregate the one or more encoded data slices portions and the other one or more encoded data slice portions to produce the encoded data slice.

6. The method of claim 1 , wherein dividing the dispersed storage error encoding process into a plurality of encoding tasks comprises:

creating a first plurality of encoding sub-task of the plurality of encoding tasks to perform a plurality of component level matrix multiplications on coefficients of the encoding matrix and data blocks of the data matrix to produce a plurality of matrix multiplied components; and

creating a second plurality of encoding sub-task of the plurality of encoding tasks to combine plurality of matrix multiplied components into the set of encoded data slices of the plurality of sets of encoded data slices.

7. A computer readable memory comprises:

a first memory section that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to:

determine dispersed storage error encoded parameters to be utilized in a dispersed storage error encoding process of a data object, wherein the dispersed storage error encoding process includes dividing the data object into a plurality of data segments and dispersed storage error encoding the plurality of the data segments into a plurality of sets of encoded data slices;

divide the dispersed storage error encoding process into a plurality of encoding tasks, wherein dividing the dispersed storage error encoding process into the plurality of encoding tasks includes:

creating a first encoding task of the plurality of encoding tasks, wherein the first encoding task is to perform a first matrix multiplication on a first portion of an encoded matrix and a first portion of a data matrix to produce one or more encoded data slices of a set of encoded data slices of the plurality of sets of encoded data slices; and

creating a second encoding task of the plurality of encoding tasks, wherein the second encoding task is to perform a second matrix multiplication on a second portion of the encoded matrix and a second portion of the data matrix to produce another one or more encoded data slices of the set of encoded data slices; and

allocate the plurality of encoding tasks to a plurality of central processing units (CPUs) available to the computing device;

a second memory section that stores operational instructions that, when executed by the plurality of CPUs, causes the plurality of CPUs to:

dispersed storage error encode the plurality of data segments into the plurality of sets of encoded data slices in accordance with the plurality of encoding tasks; and

a third memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:

send the plurality of sets of encoded data slices to storage units of the DSN for storage therein.

8. The computer readable memory of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:

determine a number of central processing units available for the dispersed storage error encoding process of the data object; and

select the plurality of CPUs from the number of central processing units.

9. The computer readable memory of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into a plurality of encoding tasks by:

creating an encoding task of the plurality of encoding tasks to be the dividing of the data object into the plurality of data segments.

10. The computer readable memory of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of encoding tasks by:

creating an encoding task of the plurality of encoding tasks to generate a data matrix for a data segment of the plurality of data segments.

11. The computer readable memory of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of encoding tasks by:

creating a first intermediate encoding task of the first encoding task to perform a first intermediate matrix multiplication on a first sub-portion of the first portion of the encoded matrix and a first sub-portion of the first portion of the data matrix to produce one or more encoded data slice portions of an encoded data slice of the one or more encoded data slices; and

creating a second intermediate encoding task of the first encoding task to perform a second intermediate matrix multiplication on a second sub-portion of the first portion of the encoded matrix and a second sub-portion of the first portion of the data matrix to produce another one or more encoded data slice portions of the encoded data slice; and

creating a third intermediate encoding task of the first encoding task to aggregate the one or more encoded data slices portions and the other one or more encoded data slice portions to produce the encoded data slice.

12. The computer readable memory of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to divide the dispersed storage error encoding process into the plurality of encoding tasks by:

creating a first plurality of encoding sub-task of the plurality of encoding tasks to perform a plurality of component level matrix multiplications on coefficients of the encoding matrix and data blocks of the data matrix to produce a plurality of matrix multiplied components; and

creating a second plurality of encoding sub-task of the plurality of encoding tasks to combine plurality of matrix multiplied components into the set of encoded data slices of the plurality of sets of encoded data slices.

Assignments (4)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Jun 11, 2025
From: BARCLAYS BANK PLC, AS ADMINISTRATIVE AGENT
To: PURE STORAGE, INC.
Reel/Frame 071558/0523 →
SECURITY INTEREST Recorded Aug 26, 2020
From: PURE STORAGE, INC.
To: BARCLAYS BANK PLC AS ADMINISTRATIVE AGENT
Reel/Frame 053867/0581 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: PURE STORAGE, INC.
Reel/Frame 049556/0288 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 28, 2016
From: BAPTIST, ANDREW D.; DHUSE, GREG R.; RESCH, JASON K.; VOLVOVSKI, ILYA
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 040434/0533 →
Continuity (3)
Continuation In Part 13270528 · Oct 11, 2011
Provisional Application 61408980 · Nov 1, 2010
Related Publication 20170075763A1 · Mar 16, 2017