Identifying optimal articulatory event-types for computer analysis of speech
A method includes, based on one or more representations of an articulatory event-type, computing, by a processor, a score quantifying an estimated degree to which an instance of the articulatory event-type indicates a state, with respect to a disease, in which the instance was produced. The method further includes storing or communicating the score for subsequent use in evaluating the state of a subject based on a test utterance produced by the subject. Other embodiments are also described.
1 . A system, comprising:
a memory configured to store program instructions; and
a processor, configured to:
load the program instructions from the memory, and
by executing the program instructions:
obtain one or more representations of a type of sound produced by articulatory organs,
compute, based on the representations, a score quantifying an estimated degree to which an instance of the type of sound, when produced by someone having a disease, indicates a state, with respect to the disease, in which the instance was produced, and
store or communicate the score for subsequent use in evaluating the state of a subject with respect to the disease, based on a test utterance produced by the subject.
2 . A method, comprising:
obtaining on one or more representations of a type of sound produced by articulatory organs;
computing, by a processor, based on the representations, a score quantifying an estimated degree to which an instance of the type of sound, when produced by someone having a disease, indicates a state, with respect to the disease, in which the instance was produced; and
storing or communicating the score for subsequent use in evaluating the state of a subject with respect to the disease, based on a test utterance produced by the subject.
3 . The method according to claim 2 , wherein the type of sound includes one or more phonemes.
4 . The method according to claim 2 , wherein the score quantifies the estimated degree to which any instance of the type of sound indicates the state.
5 . The method according to claim 4 , wherein the representations are respective segments of one or more speech samples.
6 . The method according to claim 5 ,
wherein the speech samples are first-state speech samples produced while in a first state with respect to the disease, and the segments are first-state segments, and
wherein computing the score comprises computing the score based on:
one or more same-state distances quantifying a same-state similarity of the first-state segments to each other, and
one or more cross-state distances quantifying a cross-state similarity of the first-state segments to one or more second-state segments of at least one second-state speech sample produced while in a second state with respect to the disease.
7 . The method according to claim 6 , wherein computing the score based on the same-state distances and cross-state distances comprises:
computing respective counts for multiple segments, each of which is one of the first-state segments or one of the second-state segments, by, for each of the segments:
identifying, from a set S of distances including those of the same-state distances associated with the segment and those of the cross-state distances associated with the segment, a subset S′, which includes, for a positive integer q, q smallest ones of the distances, and
computing the count for the segment as (i) a number ν of the distances in S′ that are same-state distances, or (ii) q−ν; and
computing the score based on the counts.
8 . The method according to claim 6 , wherein computing the score based on the same-state distances and cross-state distances comprises computing the score by comparing a same-state statistic of the same-state distances to a cross-state statistic of the cross-state distances.
9 . The method according to claim 5 ,
wherein the speech samples are first-state speech samples produced while in a first state with respect to the disease, and the segments are first-state segments, and
wherein computing the score comprises computing the score based on (i) multiple same-state distances between respective ones of the first-state segments and a first-state model representing the type of sound as produced while in the first state, and (ii) multiple cross-state distances between respective ones of the first-state segments and a second-state model representing the type of sound as produced while in the second state.
10 . The method according to claim 5 , wherein computing the score comprises:
obtaining respective outputs for the segments from a discriminator, each of the outputs estimating, for a different respective one of the segments, the state in which the segment was produced; and
computing the score based on a measure of accuracy of the outputs.
11 . The method according to claim 5 , wherein computing the score comprises computing the score based on respective neuronal outputs produced by a neural-network discriminator in response to evaluating, based on each of the segments, the state in which the segment was produced.
12 . The method according to claim 11 , wherein computing the score based on the neuronal outputs comprises:
computing respective score-components for the segments, each of the score-components having (i) a magnitude that is an increasing function of the neuronal output produced by the neural-network discriminator in response to the segment to which the score-component belongs, and (ii) a sign that depends on whether the evaluation, based on the segment to which the score-component belongs, is correct; and
computing the score based on the score-components.
13 . The method according to claim 4 , wherein the representations include a first-state model representing the type of sound as produced while in a first state with respect to the disease.
14 . The method according to claim 13 , wherein computing the score comprises computing the score based on a distance between the first-state model and a second-state model representing the type of sound as produced while in a second state with respect to the disease.
15 . The method according to claim 14 ,
wherein the first-state model and second-state model represent the type of sound as produced by one or more other subjects,
wherein the distance is a first distance, and
wherein computing the score comprises computing the score based on a second distance between:
a subject-specific model representing the type of sound as produced by the subject while in the first state, and
the first-state model or the second-state model.
16 . The method according to claim 13 , wherein the first-state model represents the type of sound as produced by the subject, and wherein computing the score comprises computing the score based on:
a same-state distance quantifying a same-state similarity of the first-state model to another first-state model representing the type of sound as produced by one or more other subjects while in the first state, and
a cross-state distance quantifying a cross-state similarity of the first-state model to a second-state model representing the type of sound as produced by one or more other subjects while in the second state.
17 . The method according to claim 4 ,
wherein the type of sound is a target-language type of sound in a target language,
wherein the score is a target-language score, and
wherein computing the target-language score comprises:
inputting one or more instances of the target-language type of sound to a tool configured to facilitate computing a source-language score for any instance of a source-language type of sound in a source language different from the target language, the source-language score quantifying another estimated degree to which the instance of the source-language type of sound indicates the state, with respect to the disease, in which the instance of the source-language type of sound was produced, and
computing the target-language score based on computations performed by the tool in response to the inputting.
18 . The method according to claim 17 ,
wherein the tool includes a neural network trained to predict an accuracy with which a discriminator would estimate the state based on the instance of the source-language type of sound, and
wherein computing the target-language score comprises computing the target-language score based on respective predicted accuracies output by the neural network in response to the instances of the target-language type of sound.
19 . The method according to claim 17 ,
wherein the test utterance is a target-language test utterance in the target language,
wherein the tool includes a neural-network discriminator configured to process a source-language test utterance in the source language so as to evaluate the state in which the source-language test utterance was produced, and
wherein computing the target-language score comprises computing the target-language score based on neuronal outputs produced by the neural-network discriminator in response to processing the instances of the target-language type of sound.
20 . The method according to claim 2 , wherein the representations include a segment of a speech sample representing the instance, which speech sample was produced by the subject.
21 . The method according to claim 20 , further comprising:
obtaining respective outputs, from a discriminator, for multiple training instances of the type of sound, each of the outputs estimating the state in which a different respective one of the training instances was produced; and
using the training instances, the outputs, and respective actual states in which the training instances were produced, training a neural network to predict an accuracy of the discriminator for any instance of the type of sound,
wherein computing the score comprises computing the score based on the accuracy predicted for the segment by the neural network.
22 . The method according to claim 20 , wherein computing the score comprises computing the score based on a neuronal output produced by a neural-network discriminator in response to processing the segment.
23 . A computer software product comprising a tangible non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a processor, cause the processor to:
obtain one or more representations of a type of sound produced by articulatory organs,
compute, based on the representations, a score quantifying an estimated degree to which an instance of the type of sound, when produced by someone having a disease, indicates a state, with respect to the disease, in which the instance was produced, and
store or communicate the score for subsequent use in evaluating the state of a subject with respect to the disease, based on a test utterance produced by the subject.
24 . The computer software product according to claim 23 , wherein the type of sound includes one or more phonemes.
25 . The computer software product according to claim 23 , wherein the score quantifies the estimated degree to which any instance of the type of sound indicates the state.
26 . The computer software product according to claim 25 , wherein the representations are respective segments of one or more speech samples.
27 . The computer software product according to claim 26 ,
wherein the speech samples are first-state speech samples produced while in a first state with respect to the disease, and the segments are first-state segments, and
wherein the instructions cause the processor to compute the score based on:
one or more same-state distances quantifying a same-state similarity of the first-state segments to each other, and
one or more cross-state distances quantifying a cross-state similarity of the first-state segments to one or more second-state segments of at least one second-state speech sample produced while in a second state with respect to the disease.
28 . The computer software product according to claim 27 , wherein the instructions cause the processor to compute the score based on the same-state distances and cross-state distances by:
computing respective counts for multiple segments, each of which is one of the first-state segments or one of the second-state segments, by, for each of the segments:
identifying, from a set S of distances including those of the same-state distances associated with the segment and those of the cross-state distances associated with the segment, a subset S′, which includes, for a positive integer q, q smallest ones of the distances, and
computing the count for the segment as (i) a number ν of the distances in S′ that are same-state distances, or (ii) q−ν, and
computing the score based on the counts.
29 . The computer software product according to claim 27 , wherein the instructions cause the processor to compute the score based on the same-state distances and cross-state distances by comparing a same-state statistic of the same-state distances to a cross-state statistic of the cross-state distances.
30 . The computer software product according to claim 26 ,
wherein the speech samples are first-state speech samples produced while in a first state with respect to the disease, and the segments are first-state segments, and
wherein the instructions cause the processor to compute the score based on (i) multiple same-state distances between respective ones of the first-state segments and a first-state model representing the type of sound as produced while in the first state, and (ii) multiple cross-state distances between respective ones of the first-state segments and a second-state model representing the type of sound as produced while in the second state.
31 . The computer software product according to claim 26 , wherein the instructions cause the processor to compute the score by:
obtaining respective outputs for the segments from a discriminator, each of the outputs estimating, for a different respective one of the segments, the state in which the segment was produced, and
computing the score based on a measure of accuracy of the outputs.
32 . The computer software product according to claim 26 , wherein the instructions cause the processor to compute the score based on respective neuronal outputs produced by a neural-network discriminator in response to evaluating, based on each of the segments, the state in which the segment was produced.
33 . The computer software product according to claim 32 , wherein the instructions cause the processor to compute the score based on the neuronal outputs by:
computing respective score-components for the segments, each of the score-components having (i) a magnitude that is an increasing function of the neuronal output produced by the neural-network discriminator in response to the segment to which the score-component belongs, and (ii) a sign that depends on whether the evaluation, based on the segment to which the score-component belongs, is correct, and
computing the score based on the score-components.
34 . The computer software product according to claim 25 , wherein the representations include a first-state model representing the type of sound as produced while in a first state with respect to the disease.
35 . The computer software product according to claim 34 , wherein the instructions cause the processor to compute the score based on a distance between the first-state model and a second-state model representing the type of sound as produced while in a second state with respect to the disease.
36 . The computer software product according to claim 35 ,
wherein the first-state model and second-state model represent the type of sound as produced by one or more other subjects,
wherein the distance is a first distance, and
wherein the instructions cause the processor to compute the score based on a second distance between:
a subject-specific model representing the type of sound as produced by the subject while in the first state, and
the first-state model or the second-state model.
37 . The computer software product according to claim 34 , wherein the first-state model represents the type of sound as produced by the subject, and wherein the instructions cause the processor to compute the score based on:
a same-state distance quantifying a same-state similarity of the first-state model to another first-state model representing the type of sound as produced by one or more other subjects while in the first state, and
a cross-state distance quantifying a cross-state similarity of the first-state model to a second-state model representing the type of sound as produced by one or more other subjects while in the second state.
38 . The computer software product according to claim 25 ,
wherein the type of sound is a target-language type of sound in a target language,
wherein the score is a target-language score, and
wherein the instructions cause the processor to compute the target-language score by:
inputting one or more instances of the target-language type of sound to a tool configured to facilitate computing a source-language score for any instance of a source-language type of sound in a source language different from the target language, the source-language score quantifying another estimated degree to which the instance of the source-language type of sound indicates the state, with respect to the disease, in which the instance of the source-language type of sound was produced, and
computing the target-language score based on computations performed by the tool in response to the inputting.
39 . The computer software product according to claim 38 ,
wherein the tool includes a neural network trained to predict an accuracy with which a discriminator would estimate the state based on the instance of the source-language type of sound, and
wherein the instructions cause the processor to compute the target-language score based on respective predicted accuracies output by the neural network in response to the instances of the target-language type of sound.
40 . The computer software product according to claim 38 ,
wherein the test utterance is a target-language test utterance,
wherein the tool includes a neural-network discriminator configured to process a source-language test utterance in the source language so as to evaluate the state in which the source-language test utterance was produced, and
wherein the instructions cause the processor to compute the target-language score based on neuronal outputs produced by the neural-network discriminator in response to processing the instances of the target-language type of sound.
41 . The computer software product according to claim 23 , wherein the representations include a segment of a speech sample representing the instance, which speech sample was produced by the subject.
42 . The computer software product according to claim 41 ,
wherein the instructions further cause the processor to:
obtain respective outputs, from a discriminator, for multiple training instances of the type of sound, each of the outputs estimating the state in which a different respective one of the training instances was produced, and
using the training instances, the outputs, and respective actual states in which the training instances were produced, train a neural network to predict an accuracy of the discriminator for any instance of the type of sound, and
wherein the instructions cause the processor to compute the score based on the accuracy predicted for the segment by the neural network.
43 . The computer software product according to claim 41 , wherein the instructions cause the processor to compute the score based on a neuronal output produced by a neural-network discriminator in response to processing the segment.