Methods and apparatus for parallel execution of a process
One embodiment of the invention is directed to executing a computer program that manipulates data stored in a distributed manner on a parallel server. At least one portion of the computer program treats the data from a global perspective, wherein commands to the parallel server are not specific to the manner in which the data is distributed on the parallel server. In addition, at least one portion of the computer program treats the data from a local perspective, wherein commands to the parallel server are specific to the manner in which the data is distributed on the parallel server.
1. A method of processing a computer program that manipulates a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the method comprises acts of:
(a) executing a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server, wherein the first routine is a routine that overloads a serial routine that, when executed, causes the first operation to be performed serially; and
(b) executing a second routine based on code in the computer program that causes a second request to be sent to the parallel server, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
2. The method of claim 1 , wherein the computer program is written in an interpreted programming language.
3. The method of claim 1 , wherein the computer program is written in a compiled programming language.
4. The method of claim 1 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
5. The method of claim 4 , wherein the first computer program and the second computer program are written in a same programming language.
6. The method of claim 4 , wherein the first computer program and the second computer program are written in different programming languages.
7. The method of claim 1 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
8. The method of claim 1 , wherein the first routine begins execution before the second routine.
9. The method of claim 1 , wherein the second routine begins execution before the first routine.
10. The method of claim 1 , wherein the plurality of processors are abstract processors.
11. At least one computer readable medium encoded with instructions that, when executed on a computer system, perform a method of processing a computer program that manipulates a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the method comprises acts of:
(a) executing a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server, wherein the first routine is a routine that overloads a serial routine that, when executed, causes the first operation to be performed serially; and
(b) executing a second routine based on code in the computer program that causes a second request to be sent to the parallel server, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
12. The at least one computer readable medium of claim 11 , wherein the computer program is written in an interpreted programming language.
13. The at least one computer readable medium of claim 11 , wherein the computer program is written in a compiled programming language.
14. The at least one computer readable medium of claim 11 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
15. The at least one computer readable medium of claim 14 , wherein the first computer program and the second computer program are written in a same programming language.
16. The at least one computer readable medium of claim 14 , wherein the first computer program and the second computer program are written in different programming languages.
17. The at least one computer readable medium of claim 11 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
18. The at least one computer readable medium of claim 11 , wherein the first routine begins execution before the second routine.
19. The at least one computer readable medium of claim 11 , wherein the second routine begins execution before the first routine.
20. The at least one computer readable medium of claim 11 , wherein the plurality of processors are abstract processors.
21. A computer that executes a computer program for manipulating a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the computer comprises:
an output for sending data to the parallel server; and
at least one controller, coupled to the output that:
(a) executes a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server, via the output, wherein the first routine is a routine that overloads a serial routine that, when executed, causes the first operation to be performed serially; and
(b) executes a second routine based on code in the computer program that causes a second request to be sent to the parallel server, via the output, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
22. The computer of claim 21 , wherein the computer program is written in an interpreted programming language.
23. The computer of claim 21 , wherein the computer program is written in a compiled programming language.
24. The computer of claim 21 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
25. The computer of claim 24 , wherein the first computer program and the second computer program are written in a same programming language.
26. The computer of claim 24 , wherein the first computer program and the second computer program are written in different programming languages.
27. The computer of claim 21 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
28. The computer of claim 21 , wherein the first routine begins execution before the second routine.
29. The computer of claim 21 , wherein the second routine begins execution before the first routine.
30. The computer of claim 21 , wherein the plurality of processors are abstract processors.
31. A method of processing a computer program written in an interpreted programming language that manipulates a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the method comprises acts of:
(a) executing a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server; and
(b) executing a second routine based on code in the computer program that causes a second request to be sent to the parallel server, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
32. The method of claim 31 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
33. The method of claim 32 , wherein the first computer program and the second computer program are written in a same programming language.
34. The method of claim 32 , wherein the first computer program and the second computer program are written in different programming languages.
35. The method of claim 31 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
36. The method of claim 31 , wherein the first routine begins execution before the second routine.
37. The method of claim 31 , wherein the second routine begins execution before the first routine.
38. The method of claim 31 , wherein the plurality of processors are abstract processors.
39. At least one computer readable medium encoded with instructions that, when executed on a computer system, perform a method of processing a computer program written in an interpreted programming language that manipulates a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the method comprises acts of:
(a) executing a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server; and
(b) executing a second routine based on code in the computer program that causes a second request to be sent to the parallel server, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
40. The at least one computer readable medium of claim 39 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
41. The at least one computer readable medium of claim 40 , wherein the first computer program and the second computer program are written in a same programming language.
42. The at least one computer readable medium of claim 40 , wherein the first computer program and the second computer program are written in different programming languages.
43. The at least one computer readable medium of claim 39 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
44. The at least one computer readable medium of claim 39 , wherein the first routine begins execution before the second routine.
45. The at least one computer readable medium of claim 39 , wherein the second routine begins execution before the first routine.
46. The at least one computer readable medium of claim 39 , wherein the plurality of processors are abstract processors.
47. A computer that executes a computer program written in a interpreted programming language for manipulating a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is stored on each of the plurality of processors, and wherein the computer comprises:
an output for sending data to the parallel server; and
at least one controller, coupled to the output that:
(a) executes a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server, via the output; and
(b) executes a second routine based on code in the computer program that causes a second request to be sent to the parallel server, via the output, wherein the second request specifies an abstract splitting of the data structure, and wherein the second request includes an instruction to at least one specific processor of the plurality of processors to perform a second operation on the portion of the data structure stored on the at least one specific processor.
48. The computer of claim 47 , wherein the computer program is a first computer program and the second request includes a second computer program that, when executed on the at least one specific processor, performs the second operation on the portion of the data structure stored on the at least one specific processor.
49. The computer of claim 48 , wherein the first computer program and the second computer program are written in a same programming language.
50. The computer of claim 48 , wherein the first computer program and the second computer program are written in different programming languages.
51. The computer of claim 47 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
52. The computer of claim 47 , wherein the first routine is executed before the second routine.
53. The computer of claim 47 , wherein the second routine is executed before the first routine.
54. The computer of claim 47 , wherein the plurality of processors are abstract processors.
55. A method of processing a computer program that manipulates a data structure stored on a parallel server comprising a plurality of processors, wherein a portion of the data structure is physically stored on each of the plurality of processors, and wherein the method comprises acts of:
(a) executing a first routine based on code in the computer program that causes a first request to perform a first operation on the data structure to be sent to the parallel server; and
(b) executing a second routine based on code in the computer program that causes a second request to be sent to the parallel server, wherein the second request defines an abstract splitting of the data structure that defines a plurality of abstract portions of the data structure, and wherein the second request includes an instruction to perform a second operation on at least one of the plurality of abstract portions.
56. The method of claim 55 , wherein the second routine begins execution before the first routine.
57. The method of claim 55 , wherein the parallel server includes at least one processor in addition to the plurality of processors.
58. The method of claim 55 , wherein the second operation takes as input the at least one of the plurality of abstract portions.
59. The method of claim 58 , further comprising an act of:
(c) in response to receiving the second request at the parallel server, performing the second operation on the at least one of the plurality of abstract portions without physically distributing the data structure on the parallel server in the manner of the abstract splitting.