Computing regression models

1. A computer program product, the computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, when executed by at least one processor of a computer, is configured to perform:

creating a processing data set of records, wherein each of the records contains data specific to a sub-task from a set of actual sub-tasks and contains a reference to data shared by the set of actual sub-tasks, and wherein a number of the records is equivalent to a number of the actual sub-tasks in the set of actual sub-tasks;

with each mapper in a set of mappers,

receiving one of the records of the processing data set, wherein the data specific to the sub-task is a leading-index sequence of a local search sequence, and wherein the data shared by the set of actual sub-tasks includes common-index predictors and a matrix;

generating a common-index sequence using the common predictors;

performing sweep operations on the matrix based on the leading-index sequence; and

performing sweep operations on the resulting matrix based on the common-index sequence to identify a local regression model; and

with a single reducer,

reducing the local regression model from each mapper in the set of mappers to determine a global regression model.

2. The computer program product of claim 1 , wherein the computer readable program code, when executed by the at least one processor of the computer, is configured to perform:

computing a maximum number of potential sub-tasks based on at least one characteristic of an original data set; and

computing a number of actual sub-tasks in the set of actual tasks to be executed based on a number of mappers available in the set of mappers to execute the actual sub-tasks and the maximum number of potential sub-tasks.

3. The computer program product of claim 1 , wherein the computer readable program code, when executed by the at least one processor of the computer, is configured to perform:

computing the number of actual sub-tasks, designated as M actual sub-tasks, where M=2 m , where M≦L and m<k−1, where L is the number of mappers in the set of mappers available to execute the actual sub-tasks, where k is a number of predictors.

4. The computer program product of claim 3 , wherein the computer readable program code, when executed by the at least one processor of the computer, is configured to perform:

dividing the predictors into leading-index predictors and the common-index predictors.

5. The computer program product of claim 4 , wherein the computer readable program code, when executed by the at least one processor of the computer, is configured to perform:

dividing a global search sequence into M multiple local search sequences;

for each of the local search sequences, identifying a leading-index sequence from the leading-index predictors;

creating a temporary input file with M records, wherein each of the records contains an identified leading-index sequence, and wherein each of the records contains the reference to the data shared by the set of actual sub-tasks that includes the common-index predictors and the matrix on which a sweep operation is applied.

6. The computer program product of claim 1 , wherein sending the leading-index sequence, without sending the common-index sequence, to each mapper in the set of mappers reduces network traffic.

7. The computer program product of claim 1 , wherein a Software as a Service (SaaS) is configured to perform the computer program product operations.

8. A computer system, comprising:

a processor; and

a storage device connected to the processor, wherein the storage device has stored thereon a program, and wherein the processor is configured to execute instructions of the program to perform operations, wherein the operations comprise:

creating a processing data set of records, wherein each of the records contains data specific to a sub-task from a set of actual sub-tasks and contains a reference to data shared by the set of actual sub-tasks, and wherein a number of the records is equivalent to a number of the actual sub-tasks in the set of actual sub-tasks;

with each mapper in a set of mappers,

receiving one of the records of the processing data set, wherein the data specific to the sub-task is a leading-index sequence of a local search sequence, and wherein the data shared by the set of actual sub-tasks includes common-index predictors and a matrix;

generating a common-index sequence using the common predictors;

performing sweep operations on the matrix based on the leading-index sequence; and

performing sweep operations on the resulting matrix based on the common-index sequence to identify a local regression model; and

with a single reducer,

reducing the local regression model from each mapper in the set of mappers to determine a global regression model.

9. The computer system of claim 8 , wherein the operations further comprise:

computing a maximum number of potential sub-tasks based on at least one characteristic of an original data set; and

computing a number of actual sub-tasks in the set of actual tasks to be executed based on a number of mappers in the set of mappers available to execute the actual sub-tasks and the maximum number of potential sub-tasks.

10. The computer system of claim 8 , wherein the operations further comprise:

computing the number of actual sub-tasks, designated as M actual sub-tasks, where M=2 m , where M≦L and m<k−1, where L is the number of mappers in the set of mappers available to execute the actual sub-tasks, where k is a number of predictors.

11. The computer system of claim 10 , wherein the operations further comprise:

dividing the predictors into leading-index predictors and the common-index predictors.

12. The computer system of claim 11 , wherein the operations further comprise:

dividing a global search sequence into M multiple local search sequences;

for each of the local search sequences, identifying a leading-index sequence from the leading-index predictors;

creating a temporary input file with M records, wherein each of the records contains an identified leading-index sequence, and wherein each of the records contains the reference to the data shared by the set of actual sub-tasks that includes the common-index predictors and the matrix on which a sweep operation is applied.

13. The computer system of claim 8 , wherein sending the leading-index sequence, without sending the common-index sequence, to each mapper in the set of mappers reduces network traffic.

14. The computer system of claim 8 , wherein a Software as a Service (SaaS) is provided to perform the system operations.