Method and system for automatically determining a testing order when executing a test flow
A method and system for automated multisite testing. Specifically, in one embodiment, a method is disclosed for determining a testing order of plurality of testing operations of a test flow in a multisite testing environment. The method begins by automatically walking through the test flow by performing recursion on the plurality of testing operations. Next, the method automatically assigns a plurality of relative priorities to the plurality of testing operations. The plurality of relative priorities determine the testing order used when executing each of the plurality of testing operations in said test flow. Each of the plurality of testing operations is executed only once when testing a plurality of devices under test (DUTs) in the multisite testing environment.
1. A method of determining a testing order in a multisite testing environment, comprising:
automatically walking through a plurality of testing operations in a test flow by performing recursion; and
automatically assigning a plurality of relative priorities to said plurality of testing operations, said plurality of relative priorities determining said testing order for executing each of said plurality of testing operations in said test flow only once during testing of a plurality of devices under test (DUTs) in said multisite testing environment.
2. The method of claim 1 , wherein said automatically assigning said plurality of relative priorities comprises:
assigning a first priority value of “n” to the first testing operation visited when performing said recursion;
assigning an incrementing priority value to each testing operation visited when performing said recursion by incrementing a last assigned priority value; and
determining said testing order by associating each testing operation with its highest assigned priority value.
3. The method of claim 1 , wherein said automatically assigning further comprises:
assigning said plurality of relative priorities in an inverse priority fashion, wherein a first testing operation assigned a lower priority is executed before a second testing assigned a higher priority.
4. The method of claim 1 , further comprising:
simultaneously performing said test flow following said testing order on a plurality of DUTs.
5. The method of claim 4 , further comprising:
simultaneously performing a first set of testing operations in a root test sequence on said plurality of DUTs;
branching from said root test sequence to independently perform a second set of testing operations in a diverging test sequence on a first subset of said plurality of DUTs; and
reconverging said diverging test sequence with said root test sequence at a first remerge point.
6. The method of claim 1 , wherein said automatically walking further comprises:
automatically recognizing at least one remerge point in said test flow when performing said recursion.
7. The method of claim 1 , wherein each of said plurality of DUTs comprises an integrated circuit (IC) chip.
8. A method of prioritizing testing operations for use in a multisite testing environment, comprising:
a) recursively processing a test flow based on a pass priority, said test flow comprising a plurality of interrelated testing operations, said test flow based on a plurality of passing arcs and a plurality of failing arcs associated with said plurality of testing operations;
b) assigning a first priority value of “n” to the first testing operation visited in said recursively processing said test flow; and
c) assigning an incrementing priority value to each testing operation visited in said recursively processing said test flow by incrementing a last assigned priority value;
d) determining an order of priority for said plurality of interrelated testing operations by associating each testing operation with its highest assigned priority value.
9. The method of claim 8 , wherein “n” is of value “1.”
10. The method of claim 8 , wherein said order of priority is based on an inverse priority such that lower priority values have priority over higher priority values.
11. The method of claim 8 , further comprising:
e) testing a plurality of devices under test (DUTs) in said multisite testing environment by simultaneously executing said plurality of interrelated testing operations in said test flow on said plurality of DUTs by said order of priority.
12. The method of claim 11 , further comprising:
e1) selecting a testing operation with the highest priority that has not been executed;
e2) executing said testing operation with the highest priority on DUTs reaching said testing operation with the highest priority;
e3) determining the next testing operation for each of said plurality of DUTs; and
e4) returning to e1) unless all possible testing operations have been executed.
13. The method of claim 11 , wherein each of said plurality of interrelated testing operations is executed once during said testing of said plurality of DUTs.
14. The method of claim 8 , wherein said a) comprises:
a1) following passing arcs in testing operations of said test flow until reaching an end; and
a2) returning to a diverging testing operation having the highest previously assigned priority value to follow a failing arc in testing operations of said test flow sequence, and returning to a1).
15. The method of claim 8 , wherein said c) further comprises:
c1) storing said incrementing priority value as said last assigned priority value.
16. The method of claim 8 , further comprising:
e) identifying a loop in said test flow;
f) breaking an infinite recursion in said loop; and
g) returning to said recursively processing said test flow based on a pass priority in a).
17. A computer system comprising:
a processor; and
a computer readable memory coupled to said processor and containing program instructions that, when executed, implement a method of determining a testing order in a multisite testing environment, comprising:
automatically walking through a plurality of testing operations in a test flow by performing recursion; and
automatically assigning a plurality of relative priorities to said plurality of testing operations, said plurality of relative priorities determining said testing order for executing each of said plurality of testing operations in said test flow only once during testing of a plurality of devices under test (DUTs) in said multisite testing environment.
18. The computer system of claim 17 , wherein said automatically assigning said plurality of relative priorities in said method comprises:
assigning a first priority value of “n” to the first testing operation visited when performing said recursion;
assigning an incrementing priority value to each testing operation visited when performing said recursion by incrementing a last assigned priority value; and
determining said testing order by associating each testing operation with its highest assigned priority value.
19. The computer system of claim 17 , wherein said automatically assigning in said method further comprises:
assigning said plurality of relative priorities in an inverse priority fashion, wherein a first testing operation assigned a lower priority is executed before a second testing assigned a higher priority.
20. The computer system of claim 17 , wherein said method further comprises:
simultaneously performing said test flow following said testing order on a plurality of DUTs.
21. The computer system of claim 20 , wherein said method further comprises:
simultaneously performing a first set of testing operations in a root test sequence on said plurality of DUTs;
branching from said root test sequence to independently perform a second set of testing operations in a diverging test sequence on a first subset of said plurality of DUTs; and
reconverging said diverging test sequence with said root test sequence at a first remerge point.
22. The computer system of claim 17 , wherein said automatically walking in said method further comprises:
automatically recognizing at least one remerge point in said test flow when performing said recursion.
23. The computer system of claim 17 , wherein each of said plurality of DUTs comprises an integrated circuit (IC) chip.