Automated product workflow usability assessment
An approach for assessing a user experience with a product deployment. The approach embeds markers in documentation associated with a product. The approach embeds the markers in the product, correlated with the marker positions in the documentation. The approach calculates metrics, associated with timings between markers, based on deployment of the product by a user. The approach generates a report of an experience of the user in deploying the product.
1. A computer-implemented method for assessing a user experience with a product deployment, the computer-implemented method comprising:
embedding, by one or more processors, markers in product instruction documentation for a product;
embedding, by the one or more processors, the markers in source code of the product, correlated with marker positions in the product instruction documentation;
detecting, by the one or more processors, the markers embedded in executing product code during deployment of the product, the product code comprising machine readable code derived from the source code;
tracking, by the one or more processors, detection of the markers in the executing product code through a target system during the deployment of the product in conjunction with processing of the markers in the product instruction documentation as the user follows the product instruction documentation;
calculating, by the one or more processors, metrics associated with timings between the processing of the markers in the product instruction documentation and the detecting of the markers in the executing product code; and
generating, by the one or more processors, a report of an experience of the user in deploying the product.
2. The computer-implemented method of claim 1 , wherein the markers are static markers or automatically generated markers.
3. The computer-implemented method of claim 2 , wherein the automatically generated markers comprise at least one of timestamps or session identifiers.
4. The computer-implemented method of claim 1 , wherein the markers comprise one or more of contents of code samples, special characters in a description field and labels in an orchestration system custom resource.
5. The computer-implemented method of claim 1 , wherein the metrics comprise one or more of a time interval between consecutive markers, an identity of a missing marker, a count of system errors detected while deploying the product, and a count of how many times one or more commands were executed more than once.
6. The computer-implemented method of claim 1 , wherein the report comprises one or more of longer than expected time intervals between execution of documented steps in a single workflow, system errors detected upon execution of documented steps in a single workflow and missing documented steps between steps in a single workflow.
7. The computer-implemented method of claim 1 , wherein the report is sent to an associated development team and an associated documentation team to use in updating deployment procedures.
8. A computer program product for assessing a user experience with a product deployment, the computer program product comprising:
one or more non-transitory computer readable storage media and program instructions stored on the one or more non-transitory computer readable storage media, the program instructions comprising:
program instructions to embed markers in product instruction documentation for a product;
program instructions to embed the markers in source code of the product, correlated with marker positions in the product instruction documentation;
program instructions to detect, by a processor, the markers embedded in executing product code during deployment of the product, the product code comprising machine readable code derived from the source code;
program instructions to track detection of the markers in the executing product code through a target system during deployment of the product in conjunction with processing of the markers in the product instruction documentation as the user follows the product instruction documentation;
program instructions to calculate metrics associated with timings between the processing of the markers in the product instruction documentation and the detection of the markers in the executing product code; and
program instructions to generate a report of an experience of the user in deploying the product.
9. The computer program product of claim 8 , wherein the markers are static markers or automatically generated markers.
10. The computer program product of claim 9 , wherein the automatically generated markers comprise at least one of timestamps or session identifiers.
11. The computer program product of claim 8 , wherein the markers comprise one or more of contents of code samples, special characters in a description field and labels in an orchestration system custom resource.
12. The computer program product of claim 8 , wherein the metrics comprise one or more of a time interval between consecutive markers, an identity of a missing marker, a count of system errors detected while deploying the product, and a count of how many times one or more commands were executed more than once.
13. The computer program product of claim 8 , wherein the report comprises one or more of longer than expected time intervals between execution of documented steps in a single workflow, system errors detected upon execution of documented steps in a single workflow and missing documented steps between steps in a single workflow.
14. The computer program product of claim 8 , wherein the report is sent to an associated development team and an associated documentation team to use in updating deployment procedures.
15. A computer system for assessing a user experience with a product deployment, the computer system comprising:
one or more computer processors;
one or more non-transitory computer readable storage media; and
program instructions stored on the one or more non-transitory computer readable storage media, the program instructions comprising:
program instructions to embed markers in product instruction documentation for a product;
program instructions to embed the markers in source code of the product, correlated with marker positions in the product instruction documentation;
program instructions to detect, by the one or more computer processors, the markers embedded in executing product code during deployment of the product, the product code comprising machine readable code derived from the source code;
program instructions to track detection of the markers in the executing product code through a target system during deployment of the product in conjunction with processing of the markers in the product instruction documentation as the user follows the product instruction documentation;
program instructions to calculate metrics associated with timings between the processing of the markers in the product instruction documentation and the detection of the markers in the executing product code; and
program instructions to generate a report of an experience of the user in deploying the product.
16. The computer system of claim 15 , wherein the markers are static markers or automatically generated markers and the automatically generated markers comprise at least one of timestamps or session identifiers.
17. The computer system of claim 15 , wherein the markers comprise one or more of contents of code samples, special characters in a description field and labels in an orchestration system custom resource.
18. The computer system of claim 15 , wherein the metrics comprise one or more of a time interval between consecutive markers, an identity of a missing marker, a count of system errors detected while deploying the product, and a count of how many times one or more commands were executed more than once.
19. The computer system of claim 15 , wherein the report comprises one or more of longer than expected time intervals between execution of documented steps in a single workflow, system errors detected upon execution of documented steps in a single workflow and missing documented steps between steps in a single workflow.
20. The computer system of claim 15 , wherein the report is sent to an associated development team and an associated documentation team to use in updating deployment procedures.