Contextual awareness subsystem for augmented hearing

1 . A method comprising:

receiving a recording of sound from an environment of device;

evaluating a context of the device based on the recording of sound, the evaluating including, in part:

extracting features from the recording of sound, and

including the features that are extracted in a feature vector;

applying the context as input to an adaptive learning model;

receiving, as output from the adaptive learning model, a suggestion to change a behavioral parameter of the device; and

changing the behavioral parameter of the device in accordance with the suggestion.

2 . The method of claim 1 , wherein receiving the recording of sound comprises recording sound from the environment of the device using a microphone of the device.

3 . The method of claim 1 , wherein the recording of sound is of a first time period and the evaluating of the context of the device further comprises:

retrieving context information for at least one time period preceding the first time period; and

concatenating the extracted features to the feature vector, the feature vector including additional features extracted from the context information of the at least one time period that precedes the first time period.

4 . The method of claim 1 , wherein the evaluating of the context of the device further comprises refining the context indicated by the extracted features by using a trained model to determine answers to at least one contextual question.

5 . The method of claim 1 , wherein the context of the device is indicated by one or more labels, the one or more labels including at least one of a type of location in which the device is located, an overall volume of the recorded sound, a ratio of foreground noise to background noise in the recorded sound, spectrographic information of the recorded sound, an prediction of whether a user can understand what someone else is saying, a predicted amount of mental effort the user is putting into hearing a target sound, an amount of frustration of the user, or a degree of alignment between known interests of a user and known interests a current topic of conversation.

6 . The method of claim 1 , wherein the adaptive learning model comprises an actor model and an executive model, the actor model generating a prediction that the change to the behavioral parameter will be beneficial to a user, and the executive model determining, based on currently available resources of the device, whether to surface the change to the behavioral parameter generated by the actor model.

7 . The method of claim 1 , wherein changing the behavioral parameter of the device in accordance with the suggestion comprises:

causing the device to present to a user the suggestion to change to the behavioral parameter of the device; and

responsive to user input indicating approval of the change, changing the behavioral parameter of the device.

8 . The method of claim 1 , wherein changing the behavioral parameter of the device in accordance with the suggestion comprises automatically changing the behavioral parameter of the device, in response to the suggestion, without user input.

9 . The method of claim 1 , further comprising:

receiving feedback indicating whether a user found value in the change to the behavioral parameter of the device; and

retraining the adaptive learning model based on the feedback.

10 . A non-transitory computer-readable medium comprising instructions that, when executed by a computing system, cause the computing system to perform operations including:

receiving a recording of sound from an environment of device;

evaluating a context of the device based on the recording of sound, the evaluating including, in part:

extracting features from the recording of sound, and

including the features that are extracted in a feature vector;

applying the context as input to an adaptive learning model;

receiving, as output from the adaptive learning model, a suggestion to change a behavioral parameter of the device; and

changing the behavioral parameter of the device in accordance with the suggestion.

11 . The non-transitory computer-readable medium of claim 10 , wherein receiving the recording of sound comprises recording sound from the environment of the device using a microphone of the device.

12 . The non-transitory computer-readable medium of claim 11 , wherein the recording of sound is of a first time period and the evaluating of the context of the device further comprises:

retrieving context information for at least one time period preceding the first time period; and

concatenating the extracted features to the feature vector, the feature vector including additional features extracted from the context information of the at least one time period that precedes the first time period.

13 . The non-transitory computer-readable medium of claim 11 , wherein the evaluating of the context of the device further comprises refining the context indicated by the extracted features by using a trained model to determine answers to at least one contextual question.

14 . The non-transitory computer-readable medium of claim 10 , wherein the adaptive learning model comprises an actor model and an executive model, the actor model generating a prediction that the change to the behavioral parameter will be beneficial to a user, and the executive model determining, based on currently available resources of the device, whether to surface the change to the behavioral parameter generated by the actor model.

15 . The non-transitory computer-readable medium of claim 10 , wherein changing the behavioral parameter of the device in accordance with the suggestion comprises:

causing the device to present to a user the suggestion to change to the behavioral parameter of the device; and

responsive to user input indicating approval of the change, changing the behavioral parameter of the device.

16 . The non-transitory computer-readable medium of claim 10 , wherein changing the behavioral parameter of the device in accordance with the suggestion comprises automatically changing the behavioral parameter of the device, in response to the suggestion, without user input.

17 . The non-transitory computer-readable medium of claim 10 , wherein the operations further include:

receiving feedback indicating whether a user found value in the change to the behavioral parameter of the device; and

retraining the adaptive learning model based on the feedback.

18 . A computing system comprising:

at least one processor; and

a non-transitory computer-readable medium comprising instructions that, when collectively executed by the at least one processor, cause the computing system to perform operations including:

receiving a recording of sound from an environment of device;

evaluating a context of the device based on the recording of sound, the evaluating including, in part:

extracting features from the recording of sound, and

including the features that are extracted in a feature vector;

applying the context as input to an adaptive learning model;

receiving, as output from the adaptive learning model, a suggestion to change a behavioral parameter of the device; and

changing the behavioral parameter of the device in accordance with the suggestion.