Computer-implemented system and method for distributed activity detection
A computer-implemented system and method for distributed activity detection is provided. Contextual data collected for a user performing an activity is processed on a mobile computing device. The mobile computing device extracts features from the contextual data and compares the features with a set of models. Each model represents an activity. A confidence score is assigned to each model based on the comparison with the features and the mobile computing device transmits the features to a server when the confidence scores for the models are low. The server trains a new model using the features and sends the new model to the mobile computing device.
1. A computer-implemented system for distributed activity detection, comprising:
a server comprising a hardware processor to train models;
at least one of a mobile computing device and a sensor device to:
process contextual data for a user performing an activity;
extract features from the contextual data;
compare the features with one or more of the models from the server and stored on the mobile computing device, wherein each model represents an activity;
assign a confidence score to each model based on the comparison with the features, wherein the confidence score comprises a probability that model matches the features;
receive from a user of the mobile computing device or sensor device an identifier for the features only when the confidence scores for a match of the features with each of the models are low;
transmit the identifier and features to the server only when the confidence scores for a match of the features with each of the models are low; and
the server to:
receive from the mobile computing device or sensor device, the features and the identifier on the server only when the confidence scores for each model are low,
train a new model on the server using the received features and the identifier; and
send the new model to the mobile computing device or the sensor device,
wherein providing the features and the identifier from the mobile computing device or sensor device to the server only when the confidence scores are low offsets processing expense of the server by performing activity detection on the mobile computing device or sensor device and training of new activity models on the server.
2. A system according to claim 1 , wherein the mobile computing device further extracts further features from additional contextual data received, compares the further features generated from a set of stored models, assigns a confidence score to each model based on the comparison with the features, and assigns the activity associated with the model having the highest confidence score to the activity being performed by the user.
3. A system according to claim 2 , wherein the mobile computing device further receives from the user an assignment of a different activity for the further features to replace the assigned activity associated with the model having the highest confidence score.
4. A system according to claim 1 , wherein the mobile computing device further adds the new model to the models stored on the mobile computing device.
5. A system according to claim 1 , wherein the mobile computing device further generates vectors for the features, wherein the vectors of the features are compared with the models on the mobile computing device.
6. A system according to claim 1 , wherein the mobile computing device further transmits a request to the user of the mobile computing device to identify at least one of the activity being performed and the activity still being performed.
7. A system according to claim 6 , wherein the request comprises a list of activities for selection by the user as the identifier of the activity.
8. A system according to claim 6 , wherein the mobile computing device further receives from the user a typed identity of the activity as the identifier.
9. A system according to claim 1 , further comprising:
training the new model on the server using a combination of labeled population data indexed by activity label and the user's specific contextual data.
10. A system according to claim 1 , wherein the feature extraction depends on a type of the contextual data.
11. A computer-implemented method for distributed activity detection, comprising:
processing on a mobile computing device contextual data for a user performing an activity;
extracting features from the contextual data via the mobile computing device;
comparing the features with a set of models from a server and stored on the mobile computing device, wherein each model represents an activity;
assigning a confidence score to each model based on the comparison with the features, wherein the confidence score comprises a probability that model matches the features;
receiving from a user of the mobile computing device an identifier for the features only when the confidence scores for a match of the features with each of the models are low;
transmitting the identifier and features from the mobile computing device to the server only when the confidence scores for a match of the features with each of the models are low;
receiving the features and the identifier from the mobile computing device on the server only when the confidence scores for each model are low;
training a new model on the server using the features; and
sending from the server, the new model to the mobile computing device,
wherein providing the features and the identifier from the mobile computing device to the server when the confidence scores are low offsets processing expense of the server by performing activity detection on the mobile computing device and training of new activity models on the server.
12. A method according to claim 11 , further comprising:
extracting further features from additional contextual data received;
comparing the further features with the set of models stored on the mobile computing device;
assigning a confidence score to each model based on the comparison with the features; and
assigning the activity associated with the model having the highest confidence score to the activity being performed by the user.
13. A method according to claim 12 , further comprising:
receiving from the user an assignment of a different activity for the further features to replace the assigned activity associated with the model having the highest confidence score.
14. A method according to claim 11 , further comprising:
adding the new model to the models stored on the mobile computing device.
15. A method according to claim 11 , further comprising:
generating vectors for the features, wherein the vectors of the features are compared with the models on the mobile computing device.
16. A method according to claim 11 , further comprising:
transmitting a request to the user of the mobile computing device to identify at least one of the activity being performed and the activity still being performed.
17. A method according to claim 16 , wherein the request comprises a list of activities for selection by the user as the identifier of the activity.
18. A method according to claim 16 , further comprising:
receiving from the user a typed identity of the activity as the identifier.
19. A method according to claim 11 , further comprising:
training the new model on the server using a combination of labeled population data indexed by activity label and the user's specific contextual data.
20. A method according to claim 11 , wherein the feature extraction depends on a type of the contextual data.