Evaluation output of a system or portions thereof
A method for execution by an analysis unit includes obtaining a collection of data for a particular evaluation of a system aspect. The method further includes acquiring data analysis parameters regarding the particular evaluation of the system aspect. The method further includes determining at least one of: one or more evaluation perspectives based on the data analysis parameters. The method further includes determining one or more evaluation modalities based on the data analysis parameters. The method further includes determining one or more evaluation metrics based on the data analysis parameters. The method further includes evaluating the collection of data in accordance with the at least one of the one or more evaluation metrics, the one or more evaluation perspectives, and the one or more evaluation modalities to produce one or more evaluation outputs.
1. A method comprises:
obtaining, by an analysis unit, a collection of data for a particular evaluation of a system aspect, wherein the analysis unit evaluates at least a portion of the collection of data for the particular evaluation;
acquiring, by the analysis unit, data analysis parameters regarding the particular evaluation of the system aspect, wherein the data analysis parameters include at least one of, process analysis parameter;
policy analysis parameter;
procedure analysis parameter;
certification analysis parameter;
documentation analysis parameter; and
automation analysis parameter;
determining, by the analysis unit, at least one of:
one or more evaluation perspectives based on the data analysis parameters;
one or more evaluation modalities based on the data analysis parameters; and
one or more evaluation metrics based on the data analysis parameters; and
evaluating, by the analysis unit, the collection of data in accordance with the at least one of the one or more evaluation metrics, the one or more evaluation perspectives, and the one or more evaluation modalities to produce a plurality of evaluation outputs, wherein a first evaluation output of the plurality of evaluation outputs is regarding a process based on the process analysis parameter and based on the at least one of the one or more of evaluation perspectives, the one or more evaluation modalities and the one or more evaluation metrics, and wherein a second evaluation output of the plurality of evaluation outputs is regarding a policy based on the policy analysis parameter and based on the at least one of the one or more of evaluation perspectives, the one or more evaluation modalities and the one or more evaluation metrics.
2. The method of claim 1 , wherein the system aspect comprises:
one or more system elements;
one or more system criteria; and/or
one or more system modes.
3. The method of claim 1 , wherein the evaluation perspective of the one or more evaluation perspectives comprises:
understanding;
implementation;
operation; or
self-analysis.
4. The method of claim 1 , wherein the evaluation metric of the one or more evaluation metrics comprises:
process;
policy;
procedure;
documentation;
automation; or
certification.
5. The method of claim 1 , wherein the evaluation modality of the one or more evaluation modalities comprises:
a level of sufficiency of the evaluation metric of the one or more evaluation metrics associated with the system aspect;
a level of effectiveness of the evaluation metric with respect to one or more objectives of the system aspect;
a measure of use of the evaluation metric with respect to supporting the one or more objectives of the system aspect;
a measure of appropriate use of the evaluation metric with respect to supporting the one or more objectives of the system aspect; or
a measure of consistency of use of the evaluation metric with respect to supporting the one or more objectives of the system aspect.
6. The method of claim 1 further comprises:
evaluating, by the analysis unit, the collection of data in accordance with the at least one of the one or more evaluation metrics, the one or more evaluation perspectives, and the one or more evaluation modalities to produce one or more evaluation outputs, wherein an evaluation output of the one or more evaluation outputs includes:
one or more evaluation ratings;
one or more system aspect deficiencies; or
one or more auto-corrections of the one or more system aspect deficiencies.
7. The method of claim 1 , wherein the evaluating the collection of data comprises:
when the evaluation modality of the one or more evaluation modalities is a sufficiency modality:
quantifying the data of the collection of data regarding the evaluation metric of the one or more evaluation metrics to produce quantified data;
determining a reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric; and
comparing the quantified data with the reference scale to produce the evaluation rating regarding sufficiency of the evaluation metric for the system aspect from the evaluation perspective.
8. The method of claim 1 , wherein the evaluating the collection of data comprises:
when the evaluation modality of the one or more evaluation modalities is an effectiveness modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding how effective the evaluation metric supports the one or more objectives of the system aspect from the evaluation perspective.
9. The method of claim 1 , wherein the evaluating the collection of data comprises:
when the evaluation modality of the one or more evaluation modalities is a measure of use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
10. The method of claim 1 , wherein the evaluating the collection of data comprises:
when the evaluation modality of the one or more evaluation modalities is a measure of appropriate use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding appropriate use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
11. The method of claim 1 , wherein the evaluating the collection of data comprises:
when the evaluation modality of the one or more evaluation modalities is a measure of consistency of use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding consistency of use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
12. The method of claim 1 further comprises:
determining, by the analysis unit, one or more evaluation viewpoints based on the data analysis parameters.
13. The method of claim 12 , wherein an evaluation viewpoint of the one or more evaluation viewpoints comprises:
a disclosed data viewpoint;
a discovered data viewpoint; or
a desired data viewpoint.
14. The method of claim 1 further comprises:
determining, by the analysis unit, one or more evaluation categories based on the data analysis parameters.
15. The method of claim 14 , wherein an evaluation category of the one or more evaluation categories comprises:
an identify category;
a protect category;
a detect category;
a respond category; or
a recover category.
16. A computer readable memory comprises:
a first memory section for storing operational instructions that, when executed by a computing entity, causes the computing entity to:
obtain a collection of data for a particular evaluation of a system aspect, wherein the computing entity evaluates at least a portion of the collection of data for the particular evaluation; and
acquire data analysis parameters regarding the particular evaluation of the system aspect, wherein the data analysis parameters include at least one of;
process analysis parameter;
policy analysis parameter;
procedure analysis parameter;
certification analysis parameter;
documentation analysis parameter; and
automation analysis parameter;
a second memory section for storing operational instructions that, when executed by the computing entity, causes the computing entity to:
determine at least one of:
one or more evaluation perspectives based on the data analysis parameters;
one or more evaluation modalities based on the data analysis parameters; and
one or more evaluation metrics based on the data analysis parameters; and
a third memory section for storing operational instructions that, when executed by the computing entity, causes the computing entity to:
evaluate the collection of data in accordance with the at least one of the one or more evaluation metrics, the one or more evaluation perspectives, and the one or more evaluation modalities to produce a plurality of evaluation outputs, wherein a first evaluation output of the plurality of evaluation outputs is regarding a process based on the process analysis parameter and based on the at least one of the one or more of evaluation perspectives, the one or more evaluation modalities, and the one or more evaluation metrics, and wherein a second evaluation output of the plurality of evaluation outputs is regarding a policy based on the policy analysis parameter and based on the at least one of the one or more of evaluation perspectives, the one or more evaluation modalities and the one or more evaluation metrics.
17. The computer readable memory of claim 16 , wherein the system aspect comprises:
one or more system elements;
one or more system criteria; and/or
one or more system modes.
18. The computer readable memory of claim 16 , wherein the evaluation perspective of the one or more evaluation perspectives comprises:
understanding;
implementation;
operation; or
self-analysis.
19. The computer readable memory of claim 16 , wherein the evaluation metric of the one or more evaluation metrics comprises:
process;
policy;
procedure;
documentation;
automation; or
certification.
20. The computer readable memory of claim 16 , wherein the evaluation modality of the one or more evaluation modalities comprises:
a level of sufficiency of the evaluation metric of the one or more evaluation metrics associated with the system aspect;
a level of effectiveness of the evaluation metric with respect to one or more objectives of the system aspect;
a measure of use of the evaluation metric with respect to supporting the one or more objectives of the system aspect;
a measure of appropriate use of the evaluation metric with respect to supporting the one or more objectives of the system aspect; or
a measure of consistency of use of the evaluation metric with respect to supporting the one or more objectives of the system aspect.
21. The computer readable memory of claim 16 , wherein the third memory section further storing operational instructions that, when executed by the computing entity, causes the computing entity to:
evaluate the collection of data in accordance with the at least one of the one or more evaluation metrics, the one or more evaluation perspectives, and the one or more evaluation modalities to produce one or more evaluation outputs, wherein an evaluation output of the one or more evaluation outputs includes:
one or more evaluation ratings;
one or more system aspect deficiencies; or
one or more auto-corrections of the one or more system aspect deficiencies.
22. The computer readable memory of claim 16 , wherein the third memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to evaluate the collection of data by:
when the evaluation modality of the one or more evaluation modalities is a sufficiency modality:
quantifying the data of the collection of data regarding the evaluation metric of the one or more evaluation metrics to produce quantified data;
determining a reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric; and
comparing the quantified data with the reference scale to produce the evaluation rating regarding sufficiency of the evaluation metric for the system aspect from the evaluation perspective.
23. The computer readable memory of claim 16 , wherein the third memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to evaluate the collection of data by:
when the evaluation modality of the one or more evaluation modalities is an effectiveness modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding how effective the evaluation metric supports the one or more objectives of the system aspect from the evaluation perspective.
24. The computer readable memory of claim 16 , wherein the third memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to evaluate the collection of data by:
when the evaluation modality of the one or more evaluation modalities is a measure of use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
25. The computer readable memory of claim 16 , wherein the third memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to evaluate the collection of data by:
when the evaluation modality of the one or more evaluation modalities is a measure of appropriate use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding appropriate use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
26. The computer readable memory of claim 16 , wherein the third memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to evaluate the collection of data by:
when the evaluation modality of the one or more evaluation modalities is a measure of consistency of use modality:
determining the reference scale based on the evaluation perspective of the one or more evaluation perspectives and the one or more objectives of the system aspect for the evaluation metric of the one or more evaluation metrics; and
evaluating the data of the collection of data regarding the evaluation metric with respect to the reference scale to produce the evaluation rating regarding consistency of use of the evaluation metric to support the one or more objectives of the system aspect from the evaluation perspective.
27. The computer readable memory of claim 16 , wherein the second memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to:
determine one or more evaluation viewpoints based on the data analysis parameters.
28. The computer readable memory of claim 27 , wherein an evaluation viewpoint of the one or more evaluation viewpoints comprises:
a disclosed data viewpoint;
a discovered data viewpoint; or
a desired data viewpoint.
29. The computer readable memory of claim 16 , wherein the second memory section further stores operational instructions that, when executed by the computing entity, causes the computing entity to:
determine one or more evaluation categories based on the data analysis parameters.
30. The computer readable memory of claim 29 , wherein an evaluation category of the one or more evaluation categories comprises:
an identify category;
a protect category;
a detect category;
a respond category; or
a recover category.