Method and apparatus for executing code in a distributed storage platform

1. A method comprising facilitating a processing of and/or processing (1) data and/or (2) information and/or (3) at least one signal, the (1) data and/or (2) information and/or (3) at least one signal based, at least in part, on the following:

at least one job for execution over a distributed computing platform;

at least one distributed storage platform containing one or more storage nodes stored with data required for executing the at least one job and containing one or more replicate nodes of the one or more storage nodes, wherein the data is partitioned into portions that are stored in one or more of the replicate nodes based on a predetermined scheme;

a transmission of the portions from the one or more storage nodes and from the one or more replicate nodes, based on a calculation requirement, to the distributed computing platform, one or more computing nodes of the distributed computing platform, at least one client requesting the at least one job, or a combination thereof, wherein the calculation requirement specifies a predetermined number of the portions replicated in the replicate nodes to be calculated independently and in parallel with identical processing; and

a determination that results of the identical processing are inconsistent to trigger a work load adjustment of one or more of the storage nodes, the replicate nodes, or a combination thereof.

2. A method of claim 1 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a processing of the at least one job to independently determine at the one or more storage nodes code for transforming the portions of the data to a reduced form at the one or more storage nodes, wherein the code supports the at least one job;

a processing of at least a part of the code at the one or more storage nodes to determine one or more results of the at least one job;

a processing of the one or more results to independently determine at the one or more storage nodes another code for transforming the at least one portion of the one or more results to a reduced form at the one or more storage nodes;

an aggregation of one or more transformed results via a reduce function into a combined result; and

a transmission of the one or more transformed results including the at least one portion in the reduced form, the combined result, or a combination thereof, to the distributed computing platform, one or more computing nodes of the distributed computing platform, at least one client requesting the at least one job, or a combination thereof.

3. A method of claim 2 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a processing of the code at the one or more storage nodes to determine other data,

wherein the one or more transformed results are further based, at least in part, on the other data.

4. A method of claim 2 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

at least one determination that the at least one job is a map/reduce job,

wherein the code is for causing, at least in part, a performing of at least a map phase of the map/reduce job.

5. A method of claim 4 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a processing of resource availability

information associated with the one or more storage nodes to determine priority information for executing the code.

6. A method of claim 5 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

one or more limits with respect to the resource availability information,

wherein the priority information, the executing of the code, or a combination thereof is based, at least in part, on the one or more limits.

7. A method of claim 5 , wherein the resource availability information includes, at least in part, processor availability information, bandwidth availability information, information on one or more other jobs executing at the one or more storage nodes, or a combination thereof.

8. A method of claim 2 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a selection of at least one of the one or more replicate nodes to execute the code in place of at least one of the one or more of the storage nodes requiring the work load adjustment.

9. A method of claim 2 , wherein the distributed storage platform is a distributed key-value storage system, and

the code includes one or more pointers to one or more scripts preconfigured at the at least one distributed storage platform, the one or more storage nodes, or a combination thereof.

10. A method of claim 9 , wherein

the one or more pointers represent one or more keys that are hashed to a primary storage location and one or more replicate storage locations in the storage nodes that store the at least one portion of the data, and

the one or more keys are stored in the at least one distributed storage platform, the one or more storage nodes, or a combination thereof.

11. A method of claim 1 , wherein the (1) data and/or (2) information and/or (3) at least one signal are further based, at least in part, on the following:

a relaxation of the calculation requirement of the distributed storage platform by executing the identical processing on the portions from the storage nodes and on the portions from only one of the replicate nodes.

12. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive at least one job for execution over a distributed computing platform;

determine at least one distributed storage platform containing one or more storage nodes stored with data required for executing the at least one job and containing one or more replicate nodes of the one or more storage nodes, wherein the data is partitioned into portions that are stored in one or more of the replicate nodes based on a predetermined scheme;

transmit the portions from the one or more storage nodes and from the one or more replicate nodes, based on a calculation requirement, to the distributed computing platform, one or more computing nodes of the distributed computing platform, at least one client requesting the at least one job, or a combination thereof, wherein the calculation requirement specifies a predetermined number of the portions replicated in the replicate nodes to be calculated independently and in parallel with identical processing;

determine that results of the identical processing are inconsistent to trigger a work load adjustment of one or more of the storage nodes, the replicate nodes, or a combination thereof.

13. An apparatus of claim 12 , wherein the apparatus is further caused to:

process and/or facilitate a processing of the at least one job to independently determine at one or more storage nodes of the at least one distributed storage platform code for transforming the at least a portion of the data to a reduced form at the one or more storage nodes, wherein the code supports the at least one job;

process and/or facilitate a processing of at least a part of the code at the one or more storage nodes to determine one or more results of the at least one job;

process and/or facilitate a processing of the one or more results to independently determine at the one or more storage nodes another code for transforming the at least one portion of the one or more of the results to a reduced form at the one or more storage nodes; and

cause, at least in part, a transmission of the one or more results including the at least one portion in the reduced form, the combined result, or a combination thereof, to the distributed computing platform, one or more computing nodes of the distributed computing platform, at least one client requesting the at least one job, or a combination thereof.

14. An apparatus of claim 13 , wherein the apparatus is further caused to:

process and/or facilitate a processing of the code at the one or more storage nodes to determine other data,

wherein the one or more results are further based, at least in part, on the other data.

15. An apparatus of claim 13 , wherein the apparatus is further caused to:

determine that the at least one job is a map/reduce job,

wherein the code is for causing, at least in part, a performing of at least a map phase of the map/reduce job.

16. An apparatus of claim 15 , wherein the apparatus is further caused to:

process and/or facilitate a processing of resource availability information associated with the one or more storage nodes to determine priority information for executing the code.

17. An apparatus of claim 16 , wherein the apparatus is further caused to:

determine one or more limits with respect to the resource availability information,

wherein the priority information, the executing of the code, or a combination thereof is based, at least in part, on the one or more limits.

18. An apparatus of claim 16 , wherein the resource availability information includes, at least in part, processor availability information, bandwidth availability information, information on one or more other jobs executing at the one or more storage nodes, or a combination thereof.

19. An apparatus of claim 13 , wherein the apparatus is further caused to:

determine that the one or more storage nodes include one or more replicate nodes of one of the one or more storage nodes; and

cause, at least in part, a selection of at least one of the one or more replicate nodes to execute the code.

20. An apparatus of claim 13 , wherein the code includes one or more pointers to one or more scripts preconfigured at the at least one distributed storage platform, the one or more storage nodes, or a combination thereof.