Granted Patent B2

US 10,409,863 · App. 16/236,401 · Granted Sep 10, 2019

Inventors: Henry Gabriel Victor Bequet (Cary, NC); Eric Jian Yang (Morrisville, NC); Kais Arfaoui (Raleigh, NC); Ronald Earl Stogner (Cary, NC); Partha Dutta (Morrisville, NC)

Assignee: SAS INSTITUTE INC.

G06F16/9014G06F16/90344H04L67/10

Abstract

Claims (154)

1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:

receive, by the processor and from a requesting device via a network, a request to generate a federated area (FA) package that includes at least a federated area that stores at least a job flow definition to support a performance of a job flow defined by the job flow definition, wherein:

the federated area that stores at least the job flow definition is among a plurality of federated areas maintained by one or more federated devices to store a plurality of job flow definitions, a plurality of data objects, a plurality of task routines and a plurality of instance logs that are distributed among the plurality of federated areas;

the one or more federated devices store indications of hierarchical relationships among the plurality of federated areas; and

the request specifies either the job flow or the federated area that stores at least the job flow definition;

retrieve, from among the plurality of federated areas, a first instance log of the plurality of instance logs that documents a first past performance of the job flow, wherein:

the first instance log comprises a first set of object identifiers that identify a first set of objects that are associated with the first past performance of the job flow; and

the first set of object identifiers comprises a job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the first past performance, and at least one task routine identifier that identifies at least one task routine executed in the first past performance to perform a task of the job flow;

employ the at least one data object identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object of the at least one data object employed as an input to the first past performance;

employ the at least one task routine identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a task routine of the at least one task routine executed in the first past performance;

generate the FA package to include the federated area that stores the job flow definition, along with each federated area that is identified as a federated area that stores the first instance log, that stores a data object of the at least one data object employed as an input to the first past performance, and that stores a task routine of the at least one task routine executed in the first past performance;

include a copy of the job flow definition, a copy of the first instance log, a copy of each data object of the at least one data object employed as an input to the first past performance, and a copy of each task routine of the at least one task routine executed in the first past performance in its respective federated area within the FA package;

include, in the FA package, an integrity value derived as a hash or checksum value from each federated area included in the FA package;

assign, to each federated area included in the FA package, a global federated area identifier to enable the federated area to be distinguished from the federated areas of the plurality of federated areas;

in response to inclusion of multiple federated areas in the FA package, include, in the FA package and from the indications of hierarchical relationships among the plurality of federated areas, indications of hierarchical relationships among the multiple federated areas included in the FA package to enable the relationships among the multiple federated areas included in the FA package to be maintained; and

transmit the FA package to the requesting device.

2. The apparatus of claim 1 , wherein, in response to the request comprising a first instance log identifier that identifies the first instance log to specify the job flow, the processor is caused to perform operations comprising:

employ the first instance log identifier to identify the federated area of the plurality of federated areas that stores the first instance log; and

employ the job flow identifier to identify the federated area that stores the job flow definition.

3. The apparatus of claim 1 , wherein, in response to the request comprising the job flow identifier to specify the job flow, the processor is caused to perform operations comprising:

employ the job flow identifier to identify the federated area that stores the job flow definition;

employ the job flow identifier to identify the first instance log among the plurality of instances logs as an instance log that documents the first past performance; and

employ a first instance log identifier obtained by the identification of the first instance log to identify a federated area of the plurality of federated areas that stores the first instance log.

4. The apparatus of claim 3 , wherein:

the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow; and

the processor is caused to perform operations comprising:

employ the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employ a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

include a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

5. The apparatus of claim 3 , wherein the processor is caused to perform operations comprising:

employ the job flow identifier to identify a second instance log among the plurality of instance logs as an instance log that documents a second past performance of the job flow to obtain a second instance log identifier of the second instance log, wherein:

the second instance log comprises a second set of object identifiers that identify a second set of objects that are associated with the second past performance of the job flow; and

the second set of object identifiers comprises the job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the second past performance; and at least one task routine identifier that identifies at least one task routine executed in the second past performance to perform a task of the job flow;

employ the second instance log identifier to identify a federated area of the plurality of federated areas that stores the second instance log;

employ the at least one data object identifier of the second set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object employed as an input to the second past performance;

employ the at least one task routine identifier to identify each federated area of the plurality of federated areas that stores a task routine executed in the second past performance;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores the second instance log, that stores a data object employed as an input to the second past performance, or that stores a task routine executed in the second past performance; and

include a copy of the second instance log, of each data object employed as an input to the second past performance, and of each task routine executed in the second past performance in its respective federated area within the FA package.

6. The apparatus of claim 1 , wherein, in response to the request comprising a federation area identifier that specifies the federated area that stores at least the job flow definition to specify the federated area that stores at least the job flow definition, the processor is caused to perform operations comprising:

identify the first instance log in a search of the specified federated area for instance logs;

identify the job flow definition in a search of the specified federated area for job flow definitions, wherein the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow;

employ the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employ a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

include a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

7. The apparatus of claim 1 , wherein the processor is caused, for each federated area included in the FA package, to assign, to the federated area, a human-readable federated area identifier that is identical to a human-readable federated identifier that is assigned to the corresponding federated area maintained by the one or more federated devices, wherein the global federated area identifier assigned to the federated area is randomly generated and enables the federated area to be distinguished from other federated areas around the world if the federated area is re-created from the FA package.

8. The apparatus of claim 1 , wherein multiple federated areas are included in the FA package.

9. The apparatus of claim 1 , wherein the processor is caused to include, in the FA package, a routine comprising other executable instructions to be executed by another processor of another device to:

re-create each federated area that is included in the FA package within the other device; and

use each integrity value included in the FA package to perform an integrity verification of each corresponding federated area included in the FA package.

10. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:

receive, by the processor and from a requesting device via a network, a request to generate a federated area (FA) package that includes at least a federated area that stores at least a job flow definition to support a performance of a job flow defined by the job flow definition, wherein:

the federated area that stores at least the job flow definition is among a plurality of federated areas maintained by one or more federated devices to store a plurality of job flow definitions, a plurality of data objects, a plurality of task routines and a plurality of instance logs that are distributed among the plurality of federated areas;

the one or more federated devices store indications of hierarchical relationships among the plurality of federated areas; and

the request specifies either the job flow or the federated area that stores at least the job flow definition;

retrieve, from among the plurality of federated areas, a first instance log of the plurality of instance logs that documents a first past performance of the job flow, wherein:

the first instance log comprises a first set of object identifiers that identify a first set of objects that are associated with the first past performance of the job flow; and

the first set of object identifiers comprises a job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the first past performance, and at least one task routine identifier that identifies at least one task routine executed in the first past performance to perform a task of the job flow;

employ the at least one data object identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object of the at least one data object employed as an input to the first past performance;

employ the at least one task routine identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a task routine of the at least one task routine executed in the first past performance;

generate the FA package to include the federated area that stores the job flow definition, along with each federated area that is identified as a federated area that stores the first instance log, that stores a data object of the at least one data object employed as an input to the first past performance, and that stores a task routine of the at least one task routine executed in the first past performance;

include a copy of the job flow definition, a copy of the first instance log, a copy of each data object of the at least one data object employed as an input to the first past performance, and a copy of each task routine of the at least one task routine executed in the first past performance in its respective federated area within the FA package;

include, in the FA package, an integrity value derived as a hash or checksum value from each federated area included in the FA package;

assign, to each federated area included in the FA package, a global federated area identifier to enable the federated area to be distinguished from the federated areas of the plurality of federated areas;

in response to inclusion of multiple federated areas in the FA package, include, in the FA package and from the indications of hierarchical relationships among the plurality of federated areas, indications of hierarchical relationships among the multiple federated areas included in the FA package to enable the relationships among the multiple federated areas included in the FA package to be maintained; and

transmit the FA package to the requesting device.

11. The computer-program product of claim 10 , wherein, in response to the request comprising a first instance log identifier that identifies the first instance log to specify the job flow, the processor is caused to perform operations comprising:

employ the first instance log identifier to identify the federated area of the plurality of federated areas that stores the first instance log; and

employ the job flow identifier to identify the federated area that stores the job flow definition.

12. The computer-program product of claim 10 , wherein, in response to the request comprising the job flow identifier to specify the job flow, the processor is caused to perform operations comprising:

employ the job flow identifier to identify the federated area that stores the job flow definition;

employ the job flow identifier to identify the first instance log among the plurality of instances logs as an instance log that documents the first past performance; and

employ a first instance log identifier obtained by the identification of the first instance log to identify a federated area of the plurality of federated areas that stores the first instance log.

13. The computer-program product of claim 12 , wherein:

the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow; and

the processor is caused to perform operations comprising:

employ the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employ a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

include a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

14. The computer-program product of claim 12 , wherein the processor is caused to perform operations comprising:

employ the job flow identifier to identify a second instance log among the plurality of instance logs as an instance log that documents a second past performance of the job flow to obtain a second instance log identifier of the second instance log, wherein:

the second instance log comprises a second set of object identifiers that identify a second set of objects that are associated with the second past performance of the job flow; and

the second set of object identifiers comprises the job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the second past performance; and at least one task routine identifier that identifies at least one task routine executed in the second past performance to perform a task of the job flow;

employ the second instance log identifier to identify a federated area of the plurality of federated areas that stores the second instance log;

employ the at least one data object identifier of the second set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object employed as an input to the second past performance;

employ the at least one task routine identifier to identify each federated area of the plurality of federated areas that stores a task routine executed in the second past performance;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores the second instance log, that stores a data object employed as an input to the second past performance, or that stores a task routine executed in the second past performance; and

include a copy of the second instance log, of each data object employed as an input to the second past performance, and of each task routine executed in the second past performance in its respective federated area within the FA package.

15. The computer-program product of claim 10 , wherein, in response to the request comprising a federation area identifier that specifies the federated area that stores at least the job flow definition to specify the federated area that stores at least the job flow definition, the processor is caused to perform operations comprising:

identify the first instance log in a search of the specified federated area for instance logs;

identify the job flow definition in a search of the specified federated area for job flow definitions, wherein the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow;

employ the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employ a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

include, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

include a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

16. The computer-program product of claim 10 , wherein the processor is caused, for each federated area included in the FA package, to assign, to the federated area, a human-readable federated area identifier that is identical to a human-readable federated identifier that is assigned to the corresponding federated area maintained by the one or more federated devices, wherein the global federated area identifier assigned to the federated area is randomly generated and enables the federated area to be distinguished from other federated areas around the world if the federated area is re-created from the FA package.

17. The computer-program product of claim 10 , wherein

multiple federated areas are included in the FA package.

18. The computer-program product of claim 10 , wherein the processor is caused to include, in the FA package, a routine comprising other executable instructions to be executed by another processor of another device to:

re-create each federated area that is included in the FA package within the other device; and

use each integrity value included in the FA package to perform an integrity verification of each corresponding federated area included in the FA package.

19. A computer-implemented method comprising:

receiving, by a processor and from a requesting device via a network, a request to generate a federated area (FA) package that includes at least a federated area that stores at least a job flow definition to support a performance of a job flow defined by the job flow definition, wherein:

the federated area that stores at least the job flow definition is among a plurality of federated areas maintained by one or more federated devices to store a plurality of job flow definitions, a plurality of data objects, a plurality of task routines and a plurality of instance logs that are distributed among the plurality of federated areas;

the one or more federated devices store indications of hierarchical relationships among the plurality of federated areas; and

the request specifies either the job flow or the federated area that stores at least the job flow definition;

retrieving, from among the plurality of federated areas, a first instance log of the plurality of instance logs that documents a first past performance of the job flow, wherein:

the first instance log comprises a first set of object identifiers that identify a first set of objects that are associated with the first past performance of the job flow; and

the first set of object identifiers comprises a job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the first past performance, and at least one task routine identifier that identifies at least one task routine executed in the first past performance to perform a task of the job flow;

employing, by the processor, the at least one data object identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object of the at least one data object employed as an input to the first past performance;

employing, by the processor, the at least one task routine identifier of the first set of object identifiers to identify each federated area of the plurality of federated areas that stores a task routine of the at least one task routine executed in the first past performance;

generating, by the processor, the FA package to include the federated area that stores the job flow definition, along with each federated area that is identified as a federated area that stores the first instance log, that stores a data object of the at least one data object employed as an input to the first past performance, and that stores a task routine of the at least one task routine executed in the first past performance;

including, by the processor, a copy of the job flow definition, a copy of the first instance log, a copy of each data object of the at least one data object employed as an input to the first past performance, and a copy of each task routine of the at least one task routine executed in the first past performance in its respective federated area within the FA package;

including, by the processor, in the FA package, an integrity value derived as a hash or checksum value from each federated area included in the FA package;

assigning, by the processor and to each federated area included in the FA package, a global federated area identifier to enable the federated area to be distinguished from the federated areas of the plurality of federated areas;

including, by the processor, in the FA package and from the indications of hierarchical relationships among the plurality of federated areas, indications of hierarchical relationships among the multiple federated areas included in the FA package to enable the relationships among the multiple federated areas included in the FA package to be maintained; and

transmitting, by the processor, the FA package to the requesting device via the network.

20. The computer-implemented method of claim 19 , comprising, in response to the request comprising a first instance log identifier that identifies the first instance log to specify the job flow, performing operations comprising:

employing, by the processor, the first instance log identifier to identify the federated area of the plurality of federated areas that stores the first instance log; and

employing, by the processor, the job flow identifier to identify the federated area that stores the job flow definition.

21. The computer-implemented method of claim 19 , comprising, in response to the request comprising the job flow identifier to specify the job flow, performing operations comprising:

employing, by the processor, the job flow identifier to identify the federated area that stores the job flow definition;

employing, by the processor, the job flow identifier to identify the first instance log among the plurality of instances logs as an instance log that documents the first past performance; and

employing, by the processor, a first instance log identifier obtained by the identification of the first instance log to identify a federated area of the plurality of federated areas that stores the first instance log.

22. The computer-implemented method of claim 21 , wherein:

the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow; and

the method comprise:

employing, by the processor, the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employing, by the processor, a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

including, by the processor, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

including, by the processor, a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

23. The computer-implemented method of claim 21 , comprising:

employing, by the processor, the job flow identifier to identify a second instance log among the plurality of instance logs as an instance log that documents a second past performance of the job flow to obtain a second instance log identifier of the second instance log, wherein:

the second instance log comprises a second set of object identifiers that identify a second set of objects that are associated with the second past performance of the job flow; and

the second set of object identifiers comprises the job flow identifier that identifies the job flow definition, at least one data object identifier that identifies at least one data object employed as an input to the second past performance; and at least one task routine identifier that identifies at least one task routine executed in the second past performance to perform a task of the job flow;

employing, by the processor, the second instance log identifier to identify a federated area of the plurality of federated areas that stores the second instance log;

employing, by the processor, the at least one data object identifier of the second set of object identifiers to identify each federated area of the plurality of federated areas that stores a data object employed as an input to the second past performance;

employing, by the processor, the at least one task routine identifier to identify each federated area of the plurality of federated areas that stores a task routine executed in the second past performance;

including, by the processor, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores the second instance log, that stores a data object employed as an input to the second past performance, or that stores a task routine executed in the second past performance; and

including, by the processor, a copy of the second instance log, of each data object employed as an input to the second past performance, and of each task routine executed in the second past performance in its respective federated area within the FA package.

24. The computer-implemented method of claim 19 , comprising, in response to the request comprising a federation area identifier that specifies the federated area that stores at least the job flow definition to specify the federated area that stores at least the job flow definition, performing operations comprising:

identifying, by the processor, the first instance log in a search of the specified federated area for instance logs;

identifying, by the processor, the job flow definition in a search of the specified federated area for job flow definitions, wherein the job flow definition comprises at least one flow task identifier that identifies at least one task that is to be performed by execution of at least one task routine during a performance of the job flow;

employing, by the processor, the at least one flow task identifier to identify a most recent version of each task routine of the at least one task routine to be executed during a performance of the job flow;

employing, by the processor, a task routine identifier of the most recent version of each task routine of the at least one task routine to identify each federated area of the plurality of federated areas that stores the most recent version of a task routine of the at least one task routine;

including, by the processor, in the FA package, each federated area, other than the federated area that stores the job flow definition, that is identified as a federated area that stores a most recent version of a task routine of the at least one task routine; and

including, by the processor, a copy of the most recent version of each task routine of the at least one task routine in its respective federated area within the FA package.

25. The computer-implemented method of claim 19 , comprising, for each federated area included in the FA package, assigning, by the processor, to the federated area, a human-readable federated area identifier that is identical to a human-readable federated identifier that is assigned to the corresponding federated area maintained by the one or more federated devices, wherein the global federated area identifier assigned to the federated area is randomly generated and enables the federated area to be distinguished from other federated areas around the world if the federated area is re-created from the FA package.

26. The computer-implemented method of claim 19 , wherein multiple federated areas are included in the FA package.

27. The computer-implemented method of claim 19 , comprising including, in the FA package, a routine comprising other executable instructions to be executed by another processor of another device to:

re-create each federated area that is included in the FA package within the other device; and

use each integrity value included in the FA package to perform an integrity verification of each corresponding federated area included in the FA package.

Assignments (1)

Continuity (14)

Cited By (1)

US 12,718,120