Quantification of a probability of a stroke condition occurrence via machine learning based on facial, body movements and word pronunciation
View Patent ↗The method of quantification of an occurrence probability of a stroke condition, comprises the steps of: providing, by a computer interface, instructions to a user, capturing a series of images and sounds of the user during the execution of each instruction, extracting, by a computing device, features from each series captured, providing the extracted features for each instruction to a dedicated trained machine learning model, said trained machine learning model being trained to associate an intermediate quantified value of an occurrence probability of a stroke condition, receiving the several intermediate quantified value of an occurrence probability of a stroke condition, providing the several received intermediate quantified value of an occurrence probability of a stroke condition to a trained machine learning model, said trained machine learning model being trained to associate a final quantified value of an occurrence probability of a stroke condition with several intermediate quantified values of an occurrence probability of a stroke, receiving the final quantified value of an occurrence probability of a stroke condition, and providing the final quantified value of an occurrence probability of a stroke condition.
1 . Method of quantification of an occurrence probability of a stroke condition, which comprises the steps of:
providing, by a computer interface, at least three instructions to a user, at least three said instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
capturing, by at least one capturing device, a series of images and sounds of the user during the execution of each instruction,
extracting, by a computing device, features from each series captured, in which the step of extracting comprises a step of determining, by the computing device, at least one position of at least one wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement, said at least one position being used as a feature by the trained machine learning model,
in which the step determining is configured to determine two series of positions of each wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement,
providing a first trained machine learning model, said first trained machine learning model being trained to associate an intermediate quantified value of an occurrence probability of a stroke condition with features representative of series of images and sounds of the user during the execution of instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
providing the extracted features for each instruction to said first trained machine learning model and receiving, by the computing device, the several intermediate quantified values of an occurrence probability of a stroke condition,
providing a second trained machine learning model, said second trained machine learning model being trained to classify said several intermediate quantified values using a database containing at least pre-labeled stroke condition data and other data related to a training set of patients;
providing, on a computer interface, the several received intermediate quantified value of an occurrence probability of a stroke condition to said second trained machine learning model and determining a final quantified value of an occurrence probability of a stroke condition in accordance with said several intermediate quantified values
receiving, by the computing device, the final quantified value of an occurrence probability of a stroke condition, and
providing, on a computer interface, the final quantified value of an occurrence probability of a stroke condition.
2 . Method according to claim 1 , in which the step of extracting comprises a step of transforming at least one sound captured in a spectrogram, said spectrogram being used as a feature by the trained machine learning model.
3 . Method according to claim 1 , in which the step of extracting features comprises a step of determining at least one position of at least one facial landmark of the face of the user, the method object of the present invention further comprising:
a step of stabilizing, by the computing device, of the extracted facial landmark positions during the execution of at least one facial movement by the user or during the execution of the pronunciation, by the user, of at least one group of words, and
a step of transforming, by the computing device, of the stabilized features, said transformed features being provided to a trained machine learning model.
4 . Method according to claim 1 , which comprises:
a step of training a plurality of dedicated machine learning model to associate an intermediate quantified value of an occurrence probability of a stroke condition with features representative of series of images and sounds of the user during the execution of instructions corresponding to:
an execution, by a user, of at least one facial movement,
an execution, by a user, of at least one upper-body movement,
a pronunciation, by a user, of at least one group of words, and
a step of training a machine learning model being trained to associate a final quantified value of an occurrence probability of a stroke condition with several intermediate quantified values of an occurrence probability of a stroke condition obtained from dedicated trained machine learning models.
5 . Method according to claim 1 , which comprises a step of constituting a database of series of empirically measured images and sounds of a user during the execution of instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words.
6 . Method according to claim 1 , which comprises a step of associating a stroke condition identifier with at least one series of empirically measured images and sounds of a user during the execution of instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words.
7 . Computing device of quantification of an occurrence probability of a stroke condition, which comprises:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the computing device to:
providing, by a computer interface, at least three instructions to a user, at least three said instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
capturing, by at least one capturing device, a series of images and sounds of the user during the execution of each instruction,
extracting, by a computing device, features from each series captured, in which the step of extracting comprises a step of determining, by the computing device, at least one position of at least one wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement, said at least one position being used as a feature by the trained machine learning model,
in which the step determining is configured to determine two series of positions of each wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement,
providing a first trained machine learning model, said first trained machine learning model being trained to associate an intermediate quantified value of an occurrence probability of a stroke condition with features representative of series of images and sounds of the user during the execution of instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
providing the extracted features for each instruction to said first trained machine learning model and receiving, by the computing device, the several intermediate quantified values of an occurrence probability of a stroke condition,
providing a second trained machine learning model, said second trained machine learning model being trained to classify said several intermediate quantified values using a database containing at least pre-labeled stroke condition data and other data related to a training set of patients;
providing, on a computer interface, the several received intermediate quantified value of an occurrence probability of a stroke condition to said second trained machine learning model, and determining a final quantified value of an occurrence probability of a stroke condition in accordance with said several intermediate quantified values
receiving, by the computing device, the final quantified value of an occurrence probability of a stroke condition, and
providing, on a computer interface, the final quantified value of an occurrence probability of a stroke condition.
8 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause a computing device to:
providing, by a computer interface, at least three instructions to a user, at least three said instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
capturing, by at least one capturing device, a series of images and sounds of the user during the execution of each instruction,
extracting, by a computing device, features from each series captured, in which the step of extracting comprises a step of determining, by the computing device, at least one position of at least one wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement, said at least one position being used as a feature by the trained machine learning model,
in which the step determining is configured to determine two series of positions of each wrist of the user along at least one axis in a series of images of the user during the execution, by the user, of at least one upper-body movement,
providing a first trained machine learning model, said first trained machine learning model being trained to associate an intermediate quantified value of an occurrence probability of a stroke condition with features representative of series of images and sounds of the user during the execution of instructions corresponding to:
an execution, by the user, of at least one facial movement,
an execution, by the user, of at least one upper-body movement,
a pronunciation, by the user, of at least one group of words,
providing the extracted features for each instruction to said first trained machine learning model and receiving, by the computing device, the several intermediate quantified values of an occurrence probability of a stroke condition,
providing a second trained machine learning model, said second trained machine learning model being trained to classify said several intermediate quantified values using a database containing at least pre-labeled stroke condition data and other data related to a training set of patients;
providing, on a computer interface, the several received intermediate quantified value of an occurrence probability of a stroke condition to said second trained machine learning model, and determining a final quantified value of an occurrence probability of a stroke condition in accordance with said several intermediate quantified values
receiving, by the computing device, the final quantified value of an occurrence probability of a stroke condition, and
providing, on a computer interface, the final quantified value of an occurrence probability of a stroke condition.