Learning from interactions for a spoken dialog system
In one embodiment, a semantic classifier input and a corresponding label attributed to the semantic classifier input may be obtained. A determination may be made whether the corresponding label is correct based on logged interaction data. An entry of an adaptation corpus may be generated based on a result of the determination. Operation of the semantic classifier may be adapted based on the adaptation corpus.
1. A method comprising:
obtaining a semantic classifier input for a semantic classifier and a corresponding label attributed to the semantic classifier input;
determining, via a processor, whether the corresponding label is correct based on logged interaction data, to yield a correctness result, wherein the logged interaction data comprises an input/output pair having an input and an output, the input comprising a speech recognition result in a lattice form and the output comprising one of an outcome of a call, a confirmation by a user, and a call hang-up; and
adapting operation of a spoken dialog system utilizing the semantic classifier based on the correctness result.
2. The method of claim 1 , wherein the output as a result of the input.
3. The method of claim 1 , wherein the logged interaction data further comprises:
data describing user speech; and
a non-speech user action indicating one of a negative training example and a positive training example.
4. The method of claim 1 , further comprising:
generating an entry for an adaptation corpus based on the correctness result.
5. The method of claim 4 , wherein adapting operation of the semantic classifier based on the correctness result further comprises adapting operation of the semantic classifier based on the adaptation corpus.
6. The method of claim 1 , wherein determining whether the corresponding label is correct indicates whether the corresponding label attributed by the semantic classifier is one of a correct label and an incorrect label.
7. The method of claim 1 , wherein determining whether the corresponding label is correct further comprises:
calculating a confidence level indicative of an estimated probability that the corresponding label attributed by the semantic classifier is correct to yield a calculated confidence level; and
determining that the corresponding label attributed by the semantic classifier is correct when the calculated confidence level is greater than a predetermined threshold.
8. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
obtaining a semantic classifier input for a semantic classifier and a corresponding label attributed to the semantic classifier input;
determining whether the corresponding label is correct based on logged interaction data, to yield a correctness result, wherein the logged interaction data comprises an input/output pair having an input and an output, the input comprising a speech recognition result in a lattice form and the output comprising one of an outcome of a call, a confirmation by a user, and a call hang-up; and
adapting operation of a spoken dialog system utilizing the semantic classifier based on the correctness result.
9. The system of claim 8 , wherein the output as a result of the input.
10. The system of claim 8 , wherein the logged interaction data further comprises:
data describing user speech; and
a non-speech user action indicating one of a negative training example and a positive training example.
11. The system of claim 8 , wherein the computer-readable storage medium stores additional instructions stored which, when executed by the processor, cause the processor to perform operations further comprising:
generating an entry for an adaptation corpus based on the correctness result.
12. The system of claim 11 , wherein adapting operation of the semantic classifier based on the correctness result further comprises adapting operation of the semantic classifier based on the adaptation corpus.
13. The system of claim 8 , wherein determining whether the corresponding label is correct indicates whether the corresponding label attributed by the semantic classifier is one of a correct label and an incorrect label.
14. The system of claim 8 , wherein determining whether the corresponding label is correct further comprises:
calculating a confidence level indicative of an estimated probability that the corresponding label attributed by the semantic classifier is correct to yield a calculated confidence level; and
determining that the corresponding label attributed by the semantic classifier is correct when the calculated confidence level is greater than a predetermined threshold.
15. A computer-readable storage device having instructions stored which, when executed by a processor, cause the processor to perform operations comprising:
obtaining a semantic classifier input for a semantic classifier and a corresponding label attributed to the semantic classifier input;
determining whether the corresponding label is correct based on logged interaction data, to yield a correctness result, wherein the logged interaction data comprises an input/output pair having an input and an output, the input comprising a speech recognition result in a lattice form and the output comprising one of an outcome of a call, a confirmation by a user, and a call hang-up; and
adapting operation of a spoken dialog system utilizing the semantic classifier based on the correctness result.
16. The computer-readable storage device of claim 15 , wherein the output as a result of the input.
17. The computer-readable storage device of claim 15 , wherein the logged interaction data further comprises:
data describing user speech; and
a non-speech user action indicating one of a negative training example and a positive training example.
18. The computer-readable storage device of claim 15 , wherein the computer-readable storage device stores additional instructions which, when executed by the processor, cause the processor to perform operations further comprising:
generating an entry for an adaptation corpus based on the correctness result.
19. The computer-readable storage device of claim 18 , wherein adapting operation of the semantic classifier based on the correctness result further comprises adapting operation of the semantic classifier based on the adaptation corpus.
20. The computer-readable storage device of claim 15 , wherein determining whether the corresponding label is correct indicates whether the corresponding label attributed by the semantic classifier is one of a correct label and an incorrect label.