Weakly supervised learning for improving multimodal sensing platform

1. A method of improving a machine learning model for use in context detection, the method comprising:

receiving, at one or more processors of a mobile device, a first measurement x from a first input sensing modality group of the mobile device, wherein the first input sensing modality group includes a low-power sensing source;

receiving, at one or more processors of the mobile device, a second measurement z from a second input sensing modality group of the mobile device, wherein the second input sensing modality group includes a high-power sensing source;

determining, at the one or more processors of the mobile device, a label {tilde over (y)} based on the second measurement z, wherein the label {tilde over (y)} represents a predicted user context having an associated probability of error distribution Π, wherein the associated probability of error distribution Π reflects a probability of error in statistics between the label {tilde over (y)} and an actual user context y; and

updating a machine learning model using the label {tilde over (y)} and the first measurement x, wherein the one or more processors are configured to continuously predict user context using the updated machine learning model.

2. The method of claim 1 , wherein the high-power sensing source is selected from a group consisting of: a high-resolution camera, a global positioning system (GPS), a cellular system, a Wi-Fi system, and a Bluetooth system.

3. The method of claim 1 , wherein the low-power sensing source is selected from a group consisting of: an accelerometer, a gyroscope, a magnetometer, a light sensor, a proximity sensor, a low-resolution camera, a microphone or an audio sensor, an electrocardiogram (ECG), a photoplethysmogram (PPG), a temperature sensor, a respiration sensor, a compass, and a barometer.

4. The method of claim 1 , wherein determining the label {tilde over (y)} based on the second measurement z is determined using an independently trained inference model or classifier, wherein the independently trained inference model or classifier comprises the associated probability of error Π, wherein the independently trained inference model or classifier is a deterministic algorithm for measurements obtained from the second input sensing modality group.

5. The method of claim 1 , wherein the machine learning model is configured to provide a true statistical relationship p(xly) between measurements received from the first input sensing modality group and an actual user context.

6. The method of claim 5 , wherein updating the machine learning model comprises:

calculating, at the one or more processors of the mobile device, a noise corrected estimator q(xly=s i ), wherein the noise corrected estimator q(xly=s i ) is based on an inverted probability of error distribution Π −1 and a plurality of stored training set members in a training set, the plurality of stored training set members comprising: (i) stored measurements obtained from the first input sensing modality group of the mobile device, and (ii) stored labels of predicted user contexts obtained from measurements from the second input sensing modality group of the mobile device; and

retraining the machine learning model based on the noise corrected estimator q(xly=s i ).

7. The method of claim 6 , wherein the noise corrected estimator q(xly=s i ) approximates closer to the true statistical relationship p(xly) between measurements received from the first input sensing modality group and the actual user context with an increasing number of stored training set members in the training set.

8. The method of claim 1 , wherein the machine learning model is configured to provide a true statistical relationship p(ylx) between an actual user context and measurements received from the first input sensing modality group.

9. The method of claim 8 , wherein updating the machine learning model comprises:

calculating, at the one or more processors of the mobile device, a noise corrected estimator q(y=s i lx), wherein the noise corrected estimator q(y=s i lx) is based on an inverted probability of error distribution Π −1 and a plurality of stored training set members in a training set, the plurality of stored training set members comprising: (i) stored measurements obtained from the first input sensing modality group of the mobile device, and (ii) stored labels of predicted user contexts obtained from measurements from the second input sensing modality group of the mobile device; and

retraining the machine learning model based on the noise corrected estimator q(y=s i lx).

10. The method of claim 9 , further comprising:

predicting, via the one or more processors of the mobile device, a user context using the retrained machine learning model.

11. The method of claim 10 , wherein user context is selected from a group consisting of: a physical activity, a transportation mode, an acoustic environment, a mood, and a health condition.

12. The method of claim 1 , further comprising:

storing the first measurement x and the label {tilde over (y)} locally on the mobile device as a training set member in a training set, wherein updating the machine learning model is performed locally on the mobile device.

13. The method of claim 1 , wherein updating the machine learning model occurs without user annotation.

14. The method of claim 1 , wherein updating the machine learning model comprises:

adapting the machine learning model to be personal to a user associated with the mobile device.

15. A mobile device comprising:

a low-power sensing source configured to provide a first measurement x;

a high-power sensing source configured to provide a second measurement z;

one or more processors coupled to the low-power sensing source and to the high-power sensing source, wherein the one or more processors are configured to:

receive the first measurement x from the low-power sensing source;

receive the second measurement z from the high-power sensing source;

determine a label {tilde over (y)} based on the second measurement z, wherein the label {tilde over (y)} represents a predicted user context having an associated probability of error distribution Π, wherein the associated probability of error distribution Π reflects a probability of error in statistics between the label {tilde over (y)} and an actual user context y; and

update a machine learning model using the label {tilde over (y)} and the first measurement x.

16. The mobile device of claim 15 , wherein the one or more processors are further configured to:

continuously predict user context using the updated machine learning model.

17. The mobile device of claim 15 , wherein the high-power sensing source is selected from a group consisting of: a high-resolution camera, a global positioning system (GPS), a cellular system, a Wi-Fi system, and a Bluetooth system, wherein the low-power sensing source is selected from a group consisting of: an accelerometer, a gyroscope, a magnetometer, a light sensor, a proximity sensor, a low-resolution camera, a microphone or an audio sensor, an electrocardiogram (ECG), a photoplethysmogram (PPG), a temperature sensor, a respiration sensor, a compass, and a barometer.

18. The mobile device of claim 15 , wherein the one or more processors are further configured to:

store the first measurement x and the label {tilde over (y)} locally on the mobile device as a training set member in a training set.

19. The mobile device of claim 15 , wherein the one or more processors configured to determine the label {tilde over (y)} based on the second measurement z is configured to determine the label {tilde over (y)} using an independently trained inference model or classifier, wherein the independently trained inference model or classifier comprises the associated probability of error Π, wherein the independently trained inference model or classifier is a deterministic algorithm for measurements obtained from the second input sensing modality group.