OPTIMIZER
A method and apparatus: (1) select and apply a transition from a set of first objective enhancing transitions to an initial information integration flow graph based upon how application of each transition impacts a length of a chain of nodes to produce a first set of modified information integration flow graphs that satisfy a first objective; (2) select and apply a second transition from the set of first objective transitions and a set of second objective enhancing transitions to the first set of modified information integration flow graphs to produce a second set of modified information integration flow graphs that satisfy the first objective and the second objective; and (3) identify an information integration flow graph from the first set and the second set having a lowest cost.
1 . A computer implemented method for optimizing information integration flow graphs, the method comprising:
selecting and applying a transition from a set of first objective enhancing transitions to an initial information integration flow graph based upon how application of each transition impacts a length of a chain of nodes to produce a first set of modified information integration flow graphs that satisfy a first objective;
selecting and applying a second transition from the set of first objective enhancing transitions and a set of second objective enhancing transitions to the first set of modified information integration flow graphs to produce a second set of modified information integration flow graphs that satisfy the first objective and the second objective; and
identifying an information integration flow graph from the first set and the second set having a lowest cost.
2 . The method of claim 1 , wherein the first objective is performance, wherein the set of first objective transitions consists of performance and parallelism transitions, wherein the second objective is fault tolerance and wherein the set of second objective transitions consists of fault tolerance transitions.
3 . The method of claim 2 further comprising:
identifying those portions of the modified information integration flow graph having a cost less than a predetermined cost threshold; and
applying a replication transition to each of the identified portions to produce modified information integration flow graphs of the second set.
4 . The method of claim 2 further comprising:
identifying nodes in the modified information integration flow graphs of the second set having a cost greater than a predefined cost threshold; and
adding and attaching a recovery point node to each of the identified nodes.
5 . The method of claim 4 , wherein the predetermined cost threshold comprises the average cost per node for the modified information integration flow graph.
6 . The method of claim 2 further comprising:
identifying nodes in the modified information integration flow graphs of the second set having a cost greater than a predefined cost threshold and having a maintenance cost less than a cost of redoing nodes from a previous checkpoint to the recovery point node; and
adding and attaching a recovery point node to each of the identified nodes.
7 . The method of claim 1 further comprising adding a shedding data node to a modified information integration flow graph if no modified information integration flow graphs satisfies any objective.
8 . The method of claim 7 , wherein the shedding data node is added to a chain of unary operations containing a recovery point node.
9 . The method of claim 7 , wherein the shedding data node is added to the initial information integration flow graph before an existing node based upon a cost of the existing node.
10 . The method of claim 7 , or the shedding data node is added to the initial information integration flow graph following a series of notes based upon a cost of the series of nodes.
11 . The method of claim 7 , wherein the shedding data node sheds individual data and projects out selected fields of data.
12 . The method of claim 7 , wherein the shedding data node samples incoming data and sheds tuples.
13 . An apparatus comprising:
a display;
a computer system configured to:
select and apply a transition from a set of first objective enhancing transitions to an initial information integration flow graph based upon how application of each transition impacts a length of a chain of nodes to produce a first set of modified information integration flow graphs that satisfy a first objective;
select and apply a second transition from the set of first objective transitions and a set of second objective enhancing transitions to the first set of modified information integration flow graphs to produce a second set of modified information integration flow graphs that satisfy the first objective and the second objective; and
identify an information integration flow graph from the first set and the second set having a lowest cost.
14 . The apparatus of claim 13 , wherein the first objective is performance, wherein the set of first objective transitions consists of performance and parallelism transitions, wherein the second objective is fault tolerance and wherein the set of second objective transitions consists of fault tolerance transitions.
15 . A non-transitory tangible computer readable medium comprising code configured to direct a processor to:
select and apply a transition from a set of first objective enhancing transitions to an initial information integration flow graph based upon how application of each transition impacts a length of a chain of nodes to produce a first set of modified information integration flow graphs that satisfy a first objective;
select and apply a second transition from the set of first objective transitions and a set of second objective enhancing transitions to the first set of modified information integration flow graphs to produce a second set of modified information integration flow graphs that satisfy the first objective and the second objective; and
identify an information integration flow graph from the first set and the second set having a lowest cost.