Detecting potential medically-significant errors in speech recognition results
View Patent ↗In some embodiments, the recognition results produced by a speech processing system (which may include two or more recognition results, including a top recognition result and one or more alternative recognition results) based on an analysis of a speech input, are evaluated for indications of potential significant errors. In some embodiments, the recognition results may be evaluated to determine whether a meaning of any of the alternative recognition results differs from a meaning of the top recognition result in a manner that is significant for a domain, such as the medical domain. In some embodiments, words and/or phrases that may be confused by an ASR system may be determined and associated in sets of words and/or phrases. Words and/or phrases that may be determined include those that change a meaning of a phrase or sentence when included in the phrase/sentence.
1. An apparatus comprising:
at least one processor; and
at least one computer-readable storage medium having encoded thereon executable instructions that, when executed by the at least one processor, cause the at least one processor to carry out a method of processing results of a recognition by an automatic speech recognition (ASR) system on an utterance, the results comprising two or more results identified by the ASR system as likely to be accurate recognition results for the utterance, the two or more results comprising a first result and a second result, wherein the first result is identified by the ASR system as most likely among the two or more results to be an accurate recognition result, the method comprising:
evaluating the first result using a medical fact extractor to extract a first set of one or more medical facts;
evaluating the second result using the medical fact extractor to extract a second set of one or more medical facts;
determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from a meaning of the second set of one or more medical facts; and
in response to determining that the first set of one or more medical facts has a meaning that is different in a medically significant way from the meaning of the second set of one or more medical facts, triggering presentation of an alert to a user via a user interface,
wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts comprises:
determining whether the first set comprises a fact having a fact type of a group of fact types consisting of a diagnosis fact type, a symptom fact type, and a treatment fact type;
when it is determined that the first set comprises the fact having the fact type, determining whether the second set comprises the fact having the fact type; and
in response to determining that the second set does not comprise the fact having the fact type, determining that the second set has a meaning that differs from a meaning of the first set in a medically significant way.
2. The apparatus of claim 1 , wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts further comprises:
determining whether the first set comprises a medical fact having a fact type that is marked as significant;
when it is determined that the first set comprises the medical fact having the fact type that is marked as significant, determining whether the second set comprises the medical fact; and
in response to determining that the second set does not comprise the medical fact, determining that the second set has a meaning that differs from a meaning of the first set in a medically significant way.
3. The apparatus of claim 1 , wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts further comprises:
comparing the first set to the second set to determine whether the first set of medical facts differs from the second set of medical facts; and
when it is determined that the first set differs from the second set, determining that the first set has a meaning that differs in a medically significant way from a meaning of the second set.
4. The apparatus of claim 1 , wherein the method further comprises, prior to evaluating the first result or the second result using the medical fact extractor, processing a lattice produced by the ASR system representing the two or more results identified by the ASR system to generate a string of words for the first result and a string of words for the second result.
5. The apparatus of claim 1 , wherein the results of the recognition by the ASR system comprise a plurality of results identified by the ASR system as likely to be an accurate recognition result for the speech input, and
wherein the method further comprises selecting, from the plurality of results, the two or more results to be evaluated by the medical fact extractor.
6. The apparatus of claim 5 , wherein selecting the two or more results to be evaluated from the plurality of results comprises evaluating confidence values of the two or more results.
7. At least one non-transitory computer-readable storage medium having encoded thereon executable instructions that, when executed by the at least one processor, cause the at least one processor to carry out a method of processing results of a recognition by an automatic speech recognition (ASR) system on an utterance, the results comprising two or more results identified by the ASR system as likely to be accurate recognition results for the utterance, the two or more results comprising a first result and a second result, wherein the first result is identified by the ASR system as most likely among the two or more results to be an accurate recognition result, the method comprising:
evaluating the first result using a medical fact extractor to extract a first set of one or more medical facts;
evaluating the second result using the medical fact extractor to extract a second set of one or more medical facts;
determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from a meaning of the second set of one or more medical facts; and
in response to determining that the first set of one or more medical facts has a meaning that is different in a medically significant way from the meaning of the second set of one or more medical facts, triggering presentation of an alert to a user via a user interface,
wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts comprises:
determining whether the first set comprises a fact having a fact type of a group of fact types consisting of a diagnosis fact type, a symptom fact type, and a treatment fact type;
when it is determined that the first set comprises the fact having the fact type, determining whether the second set comprises the fact having the fact type; and
in response to determining that the second set does not comprise the fact having the fact type, determining that the second set has a meaning that differs from a meaning of the first set in a medically significant way.
8. The at least one computer-readable storage medium of claim 7 , wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts further comprises:
determining whether the first set comprises a medical fact having a fact type that is marked as significant;
when it is determined that the first set comprises the medical fact having the fact type that is marked as significant, determining whether the second set comprises the medical fact; and
in response to determining that the second set does not comprise the medical fact, determining that the second set has a meaning that differs from a meaning of the first set in a medically significant way.
9. The at least one computer-readable storage medium of claim 7 , wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts further comprises:
comparing the first set to the second set to determine whether the first set of medical facts differs from the second set of medical facts; and
when it is determined that the first set differs from the second set, determining that the first set has a meaning that differs in a medically significant way from a meaning of the second set.
10. The at least one computer-readable storage medium of claim 7 , wherein the method further comprises, prior to evaluating the first result or the second result using the medical fact extractor, processing a lattice produced by the ASR system representing the two or more results identified by the ASR system to generate a string of words for the first result and a string of words for the second result.
11. The at least one computer-readable storage medium of claim 7 , wherein the results of the recognition by the ASR system comprise a plurality of results identified by the ASR system as likely to be an accurate recognition result for the speech input, and
wherein the method further comprises selecting, from the plurality of results, the two or more results to be evaluated by the medical fact extractor.
12. The at least one computer-readable storage medium of claim 11 , wherein selecting the two or more results to be evaluated from the plurality of results comprises evaluating confidence values of the two or more results.
13. A method of processing results of a recognition by an automatic speech recognition (ASR) system on an utterance, the results comprising two or more results identified by the ASR system as likely to be accurate recognition results for the utterance, the two or more results comprising a first result and a second result, wherein the first result is identified by the ASR system as most likely among the two or more results to be an accurate recognition result, the method comprising:
evaluating the first result using a medical fact extractor to extract a first set of one or more medical facts;
evaluating the second result using the medical fact extractor to extract a second set of one or more medical facts;
determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from a meaning of the second set of one or more medical facts; and
in response to determining that the first set of one or more medical facts has a meaning that is different in a medically significant way from the meaning of the second set of one or more medical facts, triggering presentation of an alert to a user via a user interface,
wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts comprises:
determining whether the first set comprises a medical fact having a fact type that is marked as significant;
when it is determined that the first set comprises the medical fact having the fact type that is marked as significant, determining whether the second set comprises the medical fact; and
in response to determining that the second set does not comprise the medical fact, determining that the second set has a meaning that differs from a meaning of the first set in a medically significant way.
14. The method of claim 13 , wherein determining whether the first set comprises a medical fact having a fact type that is marked as significant comprises determining whether the first set comprises a fact having a fact type of a group of fact types consisting of a diagnosis fact type, a symptom fact type, and a treatment fact type.
15. The method of claim 13 , wherein the determining whether the first set of one or more medical facts has a meaning that is different in a medically significant way from the second set of one or more medical facts further comprises:
comparing the first set to the second set to determine whether the first set of medical facts differs from the second set of medical facts; and
when it is determined that the first set differs from the second set, determining that the first set has a meaning that differs in a medically significant way from a meaning of the second set.
16. The method of claim 13 , further comprising, prior to evaluating the first result or the second result using the medical fact extractor, processing a lattice produced by the ASR system representing the two or more results identified by the ASR system to generate a string of words for the first result and a string of words for the second result.
17. The method of claim 13 , wherein the results of the recognition by the ASR system comprise a plurality of results identified by the ASR system as likely to be an accurate recognition result for the speech input, and
wherein the method further comprises selecting, from the plurality of results, the two or more results to be evaluated by the medical fact extractor.
18. The method of claim 13 , wherein selecting the two or more results to be evaluated from the plurality of results comprises evaluating confidence values of the two or more results.