Apparatus and method for automated evaluation of technical standards relevant to a product
A non-transitory computer readable storage medium has instructions executed by a processor to utilize natural language processing operations to produce technology standard document segments from a technology standard document. The technology standard document segments are converted to a knowledge graph characterizing relationships between the technology standard document segments. The structure of the knowledge graph, regulatory compliance rules and product information are utilized to derive large language model prompts for technology standard document segments. Output from the large language model is used to form probability ratings of requirements to test the product.
1 . A non-transitory computer readable storage medium with instructions executed by a processor to:
utilize natural language processing operations to produce technology standard document segments from a technology standard document;
convert the technology standard document segments to a knowledge graph characterizing relationships between the technology standard document segments;
utilize structure of the knowledge graph, regulatory compliance rules and product information to derive large language model prompts for technology standard document segments;
use output from the large language model to form probability ratings of requirements to test the product.
2 . The non-transitory computer readable storage medium of claim 1 wherein the natural language processing operations include utilization of regular expressions, use of Bidirectional Encoder Representations from Transformers (BERT) models, tokenization techniques and string manipulation.
3 . The non-transitory computer readable storage medium of claim 1 wherein the natural language processing operations process a table of contents to link table of contents parameters with textual clauses within the technology standard document.
4 . The non-transitory computer readable storage medium of claim 3 further comprising instructions executed by the processor to form a data frame, where each row in the data frame includes a table of contents parameter and associated textual clause.
5 . The non-transitory computer readable storage medium of claim 4 further comprising instructions executed by the processor to add a column to a row in the data frame to specify an external source identified in the associated textual clause and thereby form an augmented data frame.
6 . The non-transitory computer readable storage medium of claim 5 further comprising instructions executed by the processor to form the knowledge graph from the augmented data frame.