On-device personalization based on continual local user context learning

1 . A method performed by a user equipment (UE), comprising:

receiving, by a processor of the UE, sensor data associated with activity of a user from a plurality of inputs at a first-time instance;

determining, by the processor, a state of the user at the first-time instance based on the sensor data;

identifying, by the processor, at least one attribute of the user or the activity of the user based on the sensor data and the state of the user collected at multiple time instances including the first-time instance, wherein identifying the at least one attribute comprises using an inference model to receive selected sensor data as an input and to provide an output classifying the state of the user at the multiple time instances as an attribute of the user; and

storing in memory the state of the user collected at the multiple time instances and the at least one attribute in a format that is accessible to applications that execute on the UE and that enables such applications to perform differently based on the at least one attribute of the user.

2 . The method of claim 1 , further comprising:

selecting, by the processor, a first subset of the sensor data and a second subset of the sensor data;

determining, by the processor, an information loss for the first subset of the sensor data relative to the second subset of the sensor data based on a set of factors including frequency of change of a sensor value, sensor value differences relative to different states of the user, and whether a sensor value is determinative of one or more states of the user; and

discarding, by the processor, the first subset of the sensor data or the second subset of the sensor data based on the determined information loss,

wherein the first subset of the sensor data and the second subset of the sensor data are different and not mutually exclusive.

3 . The method of claim 2 , further comprising:

determining, by the processor, whether a predetermined balance between the determined information loss and a power cost of receiving the sensor data from the plurality of inputs is satisfied;

determining, by the processor, whether a user setting limits receipt of the sensor data from one or more of the plurality of inputs; and

adjusting, by the processor, a wake-up cycle of one or more sensors corresponding to the first subset of the sensor data or the second subset of the sensor data based on the predetermined balance and the user setting.

4 . The method of claim 2 , wherein:

selecting the first subset of the sensor data is performed randomly by activating a random subset of the plurality of inputs or selecting the second subset of the sensor data is performed randomly by de-activating a test input of the plurality of inputs;

the first subset of the sensor data relates to the plurality of inputs; and

the second subset of the sensor data relates to the plurality of inputs excluding a test input of the plurality of inputs.

5 . The method of claim 2 , wherein selecting the second subset of the sensor data comprises periodically de-activating a first input of the plurality of inputs thereby reducing a resolution of the sensor data from the first input.

6 . The method of claim 1 , wherein determining a state of the user further comprises:

matching, by the processor, the sensor data at the first-time instance with the state based on a state dictionary,

wherein the state dictionary includes a set of valid states of user context.

7 . The method of claim 1 , wherein determining a state of the user further comprises determining, by the processor, whether the sensor data at the first-time instance matches a first state in a state dictionary, and one of:

adding, by the processor, the sensor data to the state dictionary stored in memory as a first unknown state in response to determining that the sensor data at the first-time instance matches a first state in the state dictionary; or

recording, by the processor, a count of the first state in a state table that records states of the user in a first given period of time.

8 . The method of claim 7 , wherein the first unknown state is added to the state dictionary stored in memory, the method further comprising:

identifying the first unknown state as a user specific state based on an occurrence count for the first unknown state within a second given period of time;

recording, by the processor, a count of the first unknown state in the state table in response to determining that the sensor data during the first-time instance matches the first unknown state stored in the state dictionary;

requesting, via the UE, a user input to label the first unknown state; and

recording the first unknown state as a labeled state in the state dictionary based on the user input.

9 . The method of claim 7 , further comprising:

determining, by the processor, whether the first state is relevant to one or more attributes of the at least one attribute of the user; and

removing, by the processor, the first state from the state dictionary based on the count of the first state in response to whether the first state is relevant to the one or more attributes.

10 . The method of claim 7 , further comprising:

updating the inference model in an on-device retraining performed by the UE, wherein the on-device retraining is based at least in part on the state dictionary including the first unknown state.

11 . The method of claim 7 , further comprising:

determining one or more inference weights associated with the at least one attribute of the user;

transmitting only the one or more inference weights to a remote computing device to adjust the one or more inference weights based on related weights from related users, wherein users are related based at least on geolocation;

receiving one or more adjusted inference weights; and

updating the inference model based on the one or more adjusted inference weights.

12 . The method of claim 1 , further comprising training the inference model by:

querying an artificial intelligence (AI) model to complete a user profile based on a set of user profile fields;

querying the AI model to complete user activity in a given time period based on the completed user profile;

querying the AI model to complete context logs of synthetic sensor data associated with the user activity;

assembling responses to the queries into a training dataset; and

using the training dataset to derive weighting parameters associated with the inference model.

13 . The method of claim 12 , further comprising:

training the inference model based on the training dataset and the sensor data by one of:

mixing the sensor data into the training dataset;

upsampling the training dataset using the sensor data; or

querying the AI model to fill gaps in the sensor data based on the user profile.

14 . The method of claim 1 , wherein storing in the memory the state of the user collected at the multiple time instances and the at least one attribute further comprises storing the state of the user collected at the multiple time instances and the at least one attribute in a personal knowledge graph that is accessible to applications that execute on the UE and that enables such applications to perform differently based on the at least one attribute of the user.

15 . A user equipment (UE), comprising:

a plurality of sensors configured to generate sensor data;

a memory; and

a processor configured with executable instructions to:

receive sensor data associated with activity of a user from a plurality of inputs of the plurality of sensors at a first-time instance;

determine a state of the user at the first-time instance based on the sensor data;

identify at least one attribute of the user or the activity of the user based on the sensor data and the state of the user collected at multiple time instances including the first-time instance, wherein identifying the at least one attribute comprises using an inference model to receive selected sensor data as an input and to provide an output classifying the state of the user at the multiple time instances as an attribute of the user; and

store in the memory the state of the user collected at the multiple time instances and the at least one attribute in a format that is accessible to applications that execute on the UE and that enables such applications to perform differently based on the at least one attribute of the user.

16 . The UE of claim 15 , wherein the processor is further configured to:

select a first subset of the sensor data and a second subset of the sensor data;

determine an information loss for the first subset of the sensor data relative to the second subset of the sensor data based on a set of factors including frequency of change of a sensor value, sensor value differences relative to different states of the user, and whether a sensor value is determinative of one or more states of the user;

discard the first subset of the sensor data or the second subset of the sensor data based on the determined information loss; and

select the first subset of the sensor data by randomly by activating a random subset of the plurality of inputs or selecting the second subset of the sensor data is performed randomly by de-activating a test input of the plurality of inputs,

wherein the first subset of the sensor data relates to the plurality of inputs,

wherein the second subset of the sensor data relates to the plurality of inputs excluding a test input of the plurality of inputs, and

wherein the first subset of the sensor data and the second subset of the sensor data are different and not mutually exclusive.

17 . The UE of claim 16 , wherein the processor is further configured to:

determine whether a predetermined balance between the determined information loss and a power cost of receiving the sensor data from the plurality of inputs is satisfied;

determine whether a user setting limits receipt of the sensor data from one or more of the plurality of inputs; and

adjust a wake-up cycle of one or more sensors corresponding to the first subset of the sensor data or the second subset of the sensor data based on the predetermined balance and the user setting.

18 . The UE of claim 15 , wherein the processor is further configured to:

determine a state of the user by determining whether the sensor data at the first-time instance matches a first state in a state dictionary; and

perform one of:

adding the sensor data to the state dictionary stored in memory as a first unknown state in response to determining that the sensor data at the first-time instance matches a first state in the state dictionary;

identifying the first unknown state as a user specific state based on an occurrence count for the first unknown state within a second given period of time;

requesting, via the UE, a user input to label the first unknown state; and

recording the first unknown state as a labeled state in the state dictionary based on the user input; or

recording a count of the first state in a state table that records states of the user in a first given period of time.

19 . The UE of claim 15 , wherein the processor is further configured to train the inference model by:

querying an artificial intelligence (AI) model to complete a user profile based on a set of user profile fields;

querying the AI model to complete user activity in a given time period based on the completed user profile;

querying the AI model to complete context logs of synthetic sensor data associated with the user activity;

assembling responses to the queries into a training dataset;

deriving, from the training dataset, weighting parameters associated with the inference model; and

training the inference model based on the training dataset and the sensor data by one of:

mixing the sensor data into the training dataset;

upsampling the training dataset using the sensor data; or

querying the AI model to fill gaps in the sensor data based on the user profile.

20 . A non-transitory computer-readable medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations comprising:

receiving sensor data associated with activity of a user from a plurality of inputs at a first-time instance;

determining a state of the user at the first-time instance based on the sensor data;

identifying at least one attribute of the user or the activity of the user based on the sensor data and the state of the user collected at multiple time instances including the first-time instance, wherein identifying the at least one attribute comprises using an inference model to receive selected sensor data as an input and to provide an output classifying the state of the user at the multiple time instances as an attribute of the user; and

storing in memory the state of the user collected at the multiple time instances and the at least one attribute in a format that is accessible to applications that execute on a User Equipment (UE) and that enables such applications to perform differently based on the at least one attribute of the user.