Granted Patent B2

US 12,615,472 · App. 18/773,154 · Granted Apr 28, 2026

Inventors: Vishal Vijayakumar (Edina, MN); Matthew D. Kleffner (Eden Prairie, MN); Tao Zhang (Eden Prairie, MN)

Assignee: Starkey Laboratories, Inc.

H04R1/1041H04R25/505

Abstract

Claims (65)

1 . A hearing instrument comprising:

a memory configured to store a plurality of activity models, wherein each of the activity models is machine trained to determine a respective type of activity that is performed by a user of the hearing instrument; and

at least one processor configured to:

apply one or more activity models in a hierarchy of the activity models to motion data at least until a particular activity model of the plurality of activity models detects that the user is performing a particular type of activity that the particular activity model is trained to detect or until all of the activity models have been applied to the motion data and have generated a negative output, wherein the motion data is indicative of motion of the hearing instrument, and

responsive to the particular activity model determining that the user is performing the particular type of activity, output data indicating the particular type of activity is a type of activity being performed by the user.

2 . The hearing instrument of claim 1 , wherein the particular activity model is a second activity model and the particular type of activity is a second type of activity, the hierarchy of the activity models includes a first activity model trained to detect a first type of activity and the second activity model, wherein the at least one processor is configured to apply the hierarchy of the activity models by at least being configured to:

apply the first activity model to the motion data to determine whether the user is performing the first type of activity;

responsive to determining that the user is not performing the first type of activity, apply the second activity model to the motion data to determine whether the user is performing the second type of activity; and

responsive to determining that the user is performing the second type of activity, determining the second type of activity is the type of activity being performed by the user.

3 . The hearing instrument of claim 2 , wherein the plurality of activity models includes a third activity model trained to detect a first sub-type of activity that is associated with the second type of activity and a fourth activity model trained to detect a second sub-type of activity that is associated with the second type of activity, wherein the first sub-type of activity is different than the second sub-type of activity, and wherein the at least one processor is further configured to, responsive to determining that the user is performing the second type of activity:

apply the third activity model to the motion data to determine whether the user is performing the first sub-type of activity;

responsive to determining that the user is not performing the first sub-type of activity, apply the fourth activity model to the motion data to determine whether the user is performing the second sub-type of activity; and

responsive to determining that the user is performing the second sub-type of activity, determining the second sub-type of activity is the type of activity being performed by the user.

4 . The hearing instrument of claim 2 , wherein the at least one processor is further configured to:

determine, based on second motion data, that an additional type of activity being performed by the user is unknown in response to determining that the additional type of the activity is neither the first type of activity or the second type of activity; and

responsive to determining that the additional type of activity being performed by the user is unknown, output an indication of the second motion data to an edge computing device.

5 . The hearing instrument of claim 1 , wherein the hearing instrument is a first hearing instrument, and wherein the at least one processor is further configured to:

receive, from a second hearing instrument, data indicating another type of activity performed by the user;

determine whether the type of activity being performed by the user is the same as the other type of activity; and

responsive to determining that the type of activity being performed by the user is different than the other type of activity, output an indication of the motion data to an edge computing device.

6 . The hearing instrument of claim 1 , wherein the type of activity being performed by the user is a type of activity performed by the user during a first time period, and wherein the at least one processor is further configured to:

determine a type of activity performed by the user during a second time period that is within a threshold amount of time of the first time period;

determine a type of activity performed by the user during a third time period that is within the threshold amount of time of the second time period; and

responsive to determining that the type of the activity performed by the user during the first time period is different than the type of the activity performed by the user during the second time period and that the type of the activity performed by the user during the second time period is different than the type of the activity performed by the user during the third time period, perform an action to re-assign at least one of the type of activity performed by the user during the first time period, the type of activity performed by the user during the second time period, or the type of activity performed by the user during the third time period.

7 . The hearing instrument of claim 6 , wherein the at least one processor is configured to perform the action by at least being configured to:

responsive to determining that the type of the activity performed by the user during the first time period is the same as the type of the activity performed by the user during the third time period, assign the type of activity performed by the user during the second time period as the type of activity performed by the user during the first time period and the type of activity performed by the user during the third time period.

8 . The hearing instrument of claim 6 , wherein the at least one processor is configured to perform the action by at least being configured to:

output a command causing an edge computing device to determine the type of activity performed during the first time period, the type of activity performed by the user during the second time period, and the type of activity performed by the user during the third time period, wherein the command includes an indication of the motion data.

9 . The hearing instrument of claim 1 , wherein the at least one processor is further configured to:

receive data indicative of an update to an activity model of the plurality of activity models; and

update the activity model stored in the memory.

10 . The hearing instrument of claim 1 , wherein the at least one processor is further configured to determine an updated hierarchy of the plurality of activity models.

11 . The hearing instrument of claim 10 , wherein the at least one processor is further configured to determine the updated hierarchy of the plurality of activity models by at least being configured to:

determine, based on historical activity data associated with the user, a type of activity most frequently performed by the user; and

assign an activity model associated with the type of activity most frequently performed as a first activity model in the updated hierarchy.

12 . A method comprising:

receiving, by at least one processor, motion data indicative of motion of a hearing instrument;

applying, by the at least one processor, one or more activity models in a hierarchy of a plurality of activity models to the motion data at least until a particular activity model of the plurality of activity models detects that a user of the hearing instrument is performing a particular type of activity that the particular activity model is trained to detect or until all of the activity models have been applied to the motion data and have generated a negative output, wherein each of the activity models is machine trained to determine a respective type of activity that is performed by the user; and

responsive to the particular activity model determining that the user is performing the particular type of activity, outputting data indicating the particular type of activity is a type of activity being performed by the user.

13 . The method of claim 12 , wherein the particular activity model is a second activity model and the particular type of activity is a second type of activity, the hierarchy of the plurality of activity models includes a first activity model trained to detect a first type of activity, wherein applying the hierarchy of the activity models comprises:

applying, by the at least one processor, the first activity model to the motion data to determine whether the user is performing the first type of activity;

responsive to determining that the user is not performing the first type of activity, applying, by the at least one processor, the second activity model to the motion data to determine whether the user is performing the second type of activity; and

responsive to determining that the user is performing the second type of activity, determining, by the at least one processor, the second type of activity is the type of activity being performed by the user.

14 . The method of claim 13 , wherein the plurality of activity models includes a third activity model trained to detect a first sub-type of activity that is associated with the second type of activity and a fourth activity model trained to detect a second sub-type of activity that is associated with the second type of activity, wherein the first sub-type of activity is different than the second sub-type of activity, the method further comprising, responsive to determining that the user is performing the second type of activity:

applying, by the at least one processor, the third activity model to the motion data to determine whether the user is performing the first sub-type of activity;

responsive to determining that the user is not performing the first sub-type of activity, applying, by the at least one processor, the fourth activity model to the motion data to determine whether the user is performing the second sub-type of activity; and

responsive to determining that the user is performing the second sub-type of activity, determining, by the at least one processor, the second sub-type of activity is the type of activity being performed by the user.

15 . The method of claim 13 , further comprising:

determining, by the at least one processor, based on second motion data, that an additional type of activity being performed by the user is unknown in response to determining that the additional type of the activity is neither the first type of activity or the second type of activity; and

responsive to determining that the additional type of activity being performed by the user is unknown, outputting, by the at least one processor, an indication of the second motion data to an edge computing device.

16 . The method of claim 12 , wherein the hearing instrument is a first hearing instrument, the method further comprising:

receiving, by the at least one processor, from a second hearing instrument, data indicating another type of activity performed by the user;

determining, by the at least one processor, whether the type of activity being performed by the user is the same as the other type of activity; and

responsive to determining that the type of activity being performed by the user is different than the other type of activity, outputting, by the at least one processor, an indication of the motion data to an edge computing device.

17 . The method of claim 12 , wherein the type of activity being performed by the user is a type of activity performed by the user during a first time period, the method further comprising:

determining, by the at least one processor, a type of activity performed by the user during a second time period that is within a threshold amount of time of the first time period;

determining, by the at least one processor, a type of activity performed by the user during a third time period that is within the threshold amount of time of the second time period; and

responsive to determining that the type of the activity performed by the user during the first time period is different than the type of the activity performed by the user during the second time period and that the type of the activity performed by the user during the second time period is different than the type of the activity performed by the user during the third time period, performing, by the at least one processor, an action to re-assign at least one of the type of activity performed by the user during the first time period, the type of activity performed by the user during the second time period, or the type of activity performed by the user during the third time period.

18 . The method of claim 17 , wherein performing the action comprises:

responsive to determining that the type of the activity performed by the user during the first time period is the same as the type of the activity performed by the user during the third time period, assigning, by the at least one processor, the type of activity performed by the user during the second time period as the type of activity performed by the user during the first time period and the type of activity performed by the user during the third time period.

19 . The method of claim 12 , further comprising determining, by the at least one processor, an updated hierarchy of the plurality of activity models.

20 . A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor, cause the at least one processor to:

receive motion data indicative of motion of a hearing instrument;

apply one or more activity models in a hierarchy of a plurality of activity models to the motion data at least until a particular activity model of the plurality of activity models detects that a user of the hearing instrument is performing a particular type of activity that the particular activity model is trained to detect or until all of the activity models have been applied to the motion data and have generated a negative output, wherein each of the activity models is machine trained to determine a respective type of activity that is performed by the user; and

responsive to the particular activity model determining that the user is performing the particular type of activity, output data indicating the particular type of activity is a type of activity being performed by the user.

Assignments (1)

Continuity (4)

References Cited (38)