Granted Patent B2

US 12,481,926 · App. 17/705,151 · Granted Nov 25, 2025

Inventors: Maurice A. Ramirez (Lake Wales, FL); Morgan Cox (Lakewood, CO); Mark Fogarty (Clemson, SC); Robert F. Scordia (Brooklyn, NY); Nolan Donaldson (Denver, CO); Adam Stogsdill (Cypress, TX); Simon Kotchou (Phoenix, AZ); Michael V. Bivins (Orlando, FL); Allison A. Sakara (Lake Wales, FL); Karl Kelley (Springfield, TN); Mona Kelley (Springfield, TN); James Simonson (Erie, CO)

Assignee: Covid Cough, Inc.

G06N20/20G06F8/60G06N3/084G06N20/00

Abstract

Claims (118)

1 . A method comprising:

receiving, by at least one processor, a set of source signal data;

extracting, by the at least one processor, a set of signal features from the set of source signal data to produce a set of signal data signatures for each source signal data record in the set of source signal data;

generating, by the at least one processor, signal data signature features for each signal data signature of the set of signal data signatures based at least in part on the set of signal features;

training, by the at least one processor, a machine learning model based on the signal data signature features for each signal data signature of the set of signal data signatures;

deploying, by the at least one processor, a public facing machine learning model approved as a software as a medical device (SaMD) of the machine learning model for public deployment;

deploying, by the at least one processor, a development clone of the machine learning model for use in a machine learning operations (ML-Ops) pipeline of an internal development environment;

deploying, by the at least one processor, a locked clone of the machine learning model for use as a standard reference;

receiving, by the at least one processor, external feedback from the public facing model comprising at least one public facing model performance metric indicative of public facing model performance of the public facing model;

receiving, by the at least one processor, internal feedback from the development clone comprising at least one internal model performance metric indicative of internal model development performance of the development clone;

receiving, by the at least one processor, internal feedback from the locked clone comprising at least one internal locked model performance metric indicative of internal locked model performance of the locked clone;

measuring, by the at least one processor, a model drift, a context drift and a model progression associated with the public facing machine learning model, the development clone of the machine learning model, or both based at least in part on a difference between:

at least one public facing model performance metric, and

the at least one internal locked model performance metric; and

optimizing, by the at least one processor, the development clone of the machine learning model based at least in part on the model drift, the context drift and the model progression.

2 . The method of claim 1 , further comprising:

receiving, by the at least one processor, user feedback from at least one user device having the public facing machine learning model;

wherein the user feedback comprises at least one user input indicating an accuracy of at least one prediction by the public facing machine learning model;

receiving, by the at least one processor, a data log comprising at least one user signal data signature associated with the at least one prediction;

generating, by the at least one processor, a training pair comprising the at least one user input and the at least one user signal data signature; and

training, by the at least one processor, the development clone of the machine learning model based at least in part on the training pair.

3 . The method of claim 1 , further comprising:

receiving, by the at least one processor, developer feedback from at least one developer device having the locked clone of the machine learning model;

wherein the developer feedback comprises at least one developer input indicating an accuracy of at least one prediction by the locked clone of the machine learning model;

receiving, by the at least one processor, a data log comprising at least one developer signal data signature associated with the at least one prediction;

generating, by the at least one processor, a training pair comprising the at least one developer input and the at least one developer signal data signature; and

training, by the at least one processor, the development clone of the machine learning model based at least in part on the training pair.

4 . The method of claim 1 , further comprising:

generating, by the at least one processor, a second version of the development clone in response to optimizing the development clone;

wherein a first version of the development clone comprises the development clone before optimizing the development; and

storing, by the at least one processor, the first version of the development clone and the second version of the development clone in a version control database.

5 . The method of claim 4 , further comprising redeploying, by the at least one processor, the development clone of the machine learning model using the second version.

6 . The method of claim 1 , wherein the set of source signal data comprises a set of audio recordings, and the set of signal features comprise a set of source signal spectrograms associated with the set of audio recordings.

7 . The method of claim 6 , further comprising:

filtering, by the at least one processor, each audio recording of the set of audio records with a first filter module to produce a first set of filtered audio recordings;

wherein the first filter module comprises a first combination of audio filters arranged in a first sequence;

filtering, by the at least one processor, each filtered audio recording of the first set of filtered audio recordings with at least one second filter module to produce a second set of filtered audio recordings;

wherein the at least one second filter module comprises at least one second combination of audio filters arranged in at least one second sequence; and

generating, by the at least one processor, the set of source signal spectrograms based at least in part on a mel-scale spectrogram of each filtered audio recording of the second set of filtered audio recordings.

8 . A system comprising:

at least one processor configured to execute software instructions that cause the at least one processor to perform steps to:

receive a set of source signal data;

extract a set of signal features from the set of source signal data to produce a set of signal data signatures for each source signal data record in the set of source signal data;

generate signal data signature features for each signal data signature of the set of signal data signatures based at least in part on the set of signal features;

train a machine learning model based on the signal data signature features for each signal data signature of the set of signal data signatures;

deploy a public facing machine learning model approved as a software as a medical device (SaMD) of the machine learning model for public deployment;

deploy a development clone of the machine learning model for use in a machine learning operations (ML-Ops) pipeline of an internal development environment;

deploy a locked clone of the machine learning model for use as a standard reference;

receive external feedback from the public facing model comprising at least one public facing model performance metric indicative of public facing model performance of the public facing model;

receive internal feedback from the development clone comprising at least one internal model performance metric indicative of internal model development performance of the development clone;

receive internal feedback from the locked clone comprising at least one internal locked model performance metric indicative of internal locked model performance of the locked clone;

measure a model drift, a context drift and a model progression associated with the public facing machine learning model, the development clone of the machine learning model, or both based at least in part on a difference between:

at least one public facing model performance metric, and

the at least one internal locked model performance metric; and

optimize the development clone of the machine learning model based at least in part on the model drift, the context drift and the model progression.

9 . The system of claim 8 , wherein the at least one processor is further configured to execute software instructions that cause the at least one processor to perform steps to:

receive user feedback from at least one user device having the public facing machine learning model;

wherein the user feedback comprises at least one user input indicating an accuracy of at least one prediction by the public facing machine learning model;

receive a data log comprising at least one user signal data signature associated with the at least one prediction;

generate a training pair comprising the at least one user input and the at least one user signal data signature; and

train the development clone of the machine learning model based at least in part on the training pair.

10 . The system of claim 8 , wherein the at least one processor is further configured to execute software instructions that cause the at least one processor to perform steps to:

receive developer feedback from at least one developer device having the locked clone of the machine learning model;

wherein the developer feedback comprises at least one developer input indicating an accuracy of at least one prediction by the locked clone of the machine learning model;

receive a data log comprising at least one developer signal data signature associated with the at least one prediction;

generate a training pair comprising the at least one developer input and the at least one developer signal data signature; and

train the development clone of the machine learning model based at least in part on the training pair.

11 . The system of claim 8 , wherein the at least one processor is further configured to execute software instructions that cause the at least one processor to perform steps to:

generate a second version of the development clone in response to optimizing the development clone;

wherein a first version of the development clone comprises the development clone before optimizing the development; and

store the first version of the development clone and the second version of the development clone in a version control database.

12 . The system of claim 11 , wherein the at least one processor is further configured to execute software instructions that cause the at least one processor to perform steps to redeploy the development clone of the machine learning model using the second version.

13 . The system of claim 8 , wherein the set of source signal data comprises a set of audio recordings, and the set of signal features comprise a set of source signal spectrograms associated with the set of audio recordings.

14 . The system of claim 13 , wherein the at least one processor is further configured to execute software instructions that cause the at least one processor to perform steps to:

filter each audio recording of the set of audio records with a first filter module to produce a first set of filtered audio recordings;

wherein the first filter module comprises a first combination of audio filters arranged in a first sequence;

filter each filtered audio recording of the first set of filtered audio recordings with at least one second filter module to produce a second set of filtered audio recordings;

wherein the at least one second filter module comprises at least one second combination of audio filters arranged in at least one second sequence; and

generate the set of source signal spectrograms based at least in part on a mel-scale spectrogram of each filtered audio recording of the second set of filtered audio recordings.

15 . A non-transitory computer readable medium having software instructions stored thereon, the software instructions configured to cause at least one processor to perform steps comprising:

receiving a set of source signal data;

extracting a set of signal features from the set of source signal data to produce a set of signal data signatures for each source signal data record in the set of source signal data;

generating signal data signature features for each signal data signature of the set of signal data signatures based at least in part on the set of signal features;

training a machine learning model based on the signal data signature features for each signal data signature of the set of signal data signatures;

deploying a public facing machine learning model approved as a software as a medical device (SaMD) of the machine learning model for public deployment;

deploying a development clone of the machine learning model for use in a machine learning operations (ML-Ops) pipeline of an internal development environment;

deploying a locked clone of the machine learning model for use as a standard reference;

receiving external feedback from the public facing model comprising at least one public facing model performance metric indicative of public facing model performance of the public facing model;

receiving internal feedback from the development clone comprising at least one internal model performance metric indicative of internal model development performance of the development clone;

receiving internal feedback from the locked clone comprising at least one internal locked model performance metric indicative of internal locked model performance of the locked clone;

measuring a model drift, a context drift and a model progression associated with the public facing machine learning model, the development clone of the machine learning model, or both based at least in part on a difference between:

at least one public facing model performance metric, and

the at least one internal locked model performance metric; and

optimizing the development clone of the machine learning model based at least in part on the model drift, the context drift and the model progression.

16 . The non-transitory computer readable medium of claim 15 , wherein the steps further comprise:

receiving user feedback from at least one user device having the public facing machine learning model;

wherein the user feedback comprises at least one user input indicating an accuracy of at least one prediction by the public facing machine learning model;

receiving a data log comprising at least one user signal data signature associated with the at least one prediction;

generating a training pair comprising the at least one user input and the at least one user signal data signature; and

training the development clone of the machine learning model based at least in part on the training pair.

17 . The non-transitory computer readable medium of claim 15 , wherein the steps further comprise:

receiving developer feedback from at least one developer device having the locked clone of the machine learning model;

wherein the developer feedback comprises at least one developer input indicating an accuracy of at least one prediction by the locked clone of the machine learning model;

receiving a data log comprising at least one developer signal data signature associated with the at least one prediction;

generating a training pair comprising the at least one developer input and the at least one developer signal data signature; and

training the development clone of the machine learning model based at least in part on the training pair.

18 . The non-transitory computer readable medium of claim 15 , wherein the steps further comprise:

generating a second version of the development clone in response to optimizing the development clone;

wherein a first version of the development clone comprises the development clone before optimizing the development; and

storing the first version of the development clone and the second version of the development clone in a version control database.

19 . The non-transitory computer readable medium of claim 18 , wherein the steps further comprise redeploying the development clone of the machine learning model using the second version.

20 . The non-transitory computer readable medium of claim 15 , wherein the set of source signal data comprises a set of audio recordings, and the set of signal features comprise a set of source signal spectrograms associated with the set of audio recordings; and

wherein the steps further comprise:

filtering each audio recording of the set of audio records with a first filter module to produce a first set of filtered audio recordings;

wherein the first filter module comprises a first combination of audio filters arranged in a first sequence;

filtering each filtered audio recording of the first set of filtered audio recordings with at least one second filter module to produce a second set of filtered audio recordings;

wherein the at least one second filter module comprises at least one second combination of audio filters arranged in at least one second sequence; and

generating the set of source signal spectrograms based at least in part on a mel-scale spectrogram of each filtered audio recording of the second set of filtered audio recordings.

Assignments (1)

Continuity (2)

References Cited (100)

Cited By (1)

US 12,645,940