Method and apparatus for using a generative large language model for audiological fitting and device control

1 . A fitting apparatus for a hearing device comprising:

a processor; and

a memory having stored thereon:

non-transitory computer readable instructions executable by the processor, and

a plurality of embedding vectors and related solutions for possible hearing issues with the hearing device or a computing device,

and further wherein, upon execution of the instructions, the fitting apparatus is configured to:

receive, via a user interface of the hearing device or the computing device, a first signal indicative of a natural language input, wherein the natural language input is indicative of an issue experienced by a user with the hearing device or the computing device;

determine an embedding based on the natural language input, wherein the embedding comprises embeddings related to multiple dimensions;

generate, based on the embedding, a first embedding vector, wherein the first embedding vector is indicative of values associated with each of the embeddings related to multiple dimensions;

determine a closest match between the first embedding vector and an embedding vector of the plurality of embedding vectors; and

send, to the user interface of the hearing device or the computing device, a second signal indicative of a reply output, the reply output comprising information related to a solution of the related solutions for the issue experienced by the user.

2 . The fitting apparatus for the hearing device of claim 1 , wherein the embedding is a first embedding, and wherein the fitting apparatus is further configured to, upon execution of the instructions:

receive a third signal indicative of a setting of the hearing device or the computing device; and

determine a second embedding based on the setting, wherein the first embedding vector is generated based on both the first embedding and the second embedding.

3 . The fitting apparatus for the hearing device of claim 2 , wherein the setting of the hearing device or the computing device comprises at least one of:

a fitting parameter set of a hearing device;

a type of the hearing device;

a coupling setting or type of the hearing device;

a feedback threshold of the hearing device;

a real ear transfer function setting of the hearing device;

a target gain of the hearing device;

actual gain of the hearing device;

an actuator setting of the hearing device;

actual feedback stability of the hearing device; or

a hardware test parameter of the hearing device.

4 . The fitting apparatus for the hearing device of claim 2 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

receive a fourth signal indicative of a metric measured by the hearing device or the computing device; and

determine a third embedding based on the setting, wherein the first embedding vector is generated based on both the first embedding, the second embedding, and the third embedding.

5 . The fitting apparatus for the hearing device of claim 4 , wherein the metric comprises at least one of:

a signal level;

a noise floor estimation;

a signal-to-noise ratio;

a classification of a sound source; or

an estimated listening intention.

6 . The fitting apparatus for the hearing device of claim 1 , wherein the computing device comprises at least one of a user device or an external device.

7 . The fitting apparatus for the hearing device of claim 1 , wherein the natural language input further comprises temporal information related to the issue experienced by the user, the temporal information indicative of whether the issue is being currently experienced or was previously experienced by the user.

8 . The fitting apparatus for the hearing device of claim 1 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

receive, via the user interface of the hearing device or the computing device, a third signal indicative of a confirmatory response to the reply output, the confirmatory response indicative of a user assent to implementing the solution; and

implement the solution in the hearing device or the computing device, or send an instruction to implement the solution to the hearing device or the computing device.

9 . The fitting apparatus for the hearing device of claim 1 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

send, to the user interface of the hearing device or the computing device, a third signal indicative of a request to confirm an efficacy of the solution for the issue; and

receive, via the user interface of the hearing device or the computing device, a fourth signal indicative of a confirmatory response to the request.

10 . The fitting apparatus for the hearing device of claim 9 , wherein the fitting apparatus is further configured to, upon execution of the instructions, update the plurality of embedding vectors and/or the related solutions based on the confirmatory response to the request.

11 . The fitting apparatus for the hearing device of claim 1 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

send, prior to sending the second signal indicative of the reply output, a third signal indicative of a follow-up request to the user interface of the hearing device or the computing device, wherein the follow-up request requests more information related to the issue experienced by the user; and

receive, via the user interface of the hearing device or the computing device, a fourth signal indicative of a follow-up answer to the follow-up request,

wherein the first embedding vector is further determined at least in part based on the follow-up request.

12 . The fitting apparatus for the hearing device of claim 1 , wherein the natural language input comprises at least one of:

text entered by the user through a keyboard of the user interface of the hearing device or the computing device; or

speech spoken by the user and sensed by a sound detector of the hearing device or the computing device.

13 . The fitting apparatus for the hearing device of claim 1 , wherein each of the plurality of embedding vectors and the related solutions are manually defined based on previously identified problems with hearing devices and the previously identified problems' respective solutions.

14 . The fitting apparatus for the hearing device of claim 1 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

determine, based on first signal indicative of the natural language input, a translation of the natural language input; and

send, to a computing device associated with a health care provider, a third signal indicative of the translation.

15 . A method for fitting a hearing device comprising:

receiving, by a processor via a user interface of the hearing device or a computing device, a first signal indicative of a natural language input, wherein the natural language input is indicative of an issue experienced by a user with the hearing device or the computing device;

determining, by the processor, a first embedding based on the natural language input, wherein the first embedding is related to a first dimension of multiple dimensions;

receiving, by the processor, a second signal indicative of:

a setting of the hearing device or the computing device, or

a metric measured by the hearing device or the computing device;

determining, by the processor, a second embedding based on the setting or the metric, wherein the second embedding is related to a second dimension of multiple dimensions;

generating, by the processor based on the first embedding and the second embedding, a first embedding vector, wherein the first embedding vector is indicative of values associated with each of the first embedding related to the first dimension and the second embedding related to the second dimension;

determining, by the processor, a closest match between the first embedding vector and an embedding vector of an embedding vector of a plurality of embedding vectors, wherein each of the plurality of embedding vectors is associated with one of a plurality of related solutions for possible hearing issues with the hearing device or the computing device; and

sending, by the processor to the user interface of the hearing device or the computing device, a third signal indicative of a reply output, the reply output comprising information related to a solution of the related solutions for the issue experienced by the user.

16 . A fitting apparatus for a hearing device comprising:

a processor; and

a memory having stored thereon:

non-transitory computer readable instructions executable by the processor, and

a plurality of embedding vectors and related solutions for possible hearing issues with the hearing device or a computing device,

and further wherein, upon execution of the instructions, the fitting apparatus is configured to:

receive, via a user interface of the hearing device or the computing device, a first signal indicative of a natural language input, wherein the natural language input is indicative of an issue experienced by a user with the hearing device or the computing device;

determine an embedding based on the natural language input;

send, after receiving the first signal indicative of the natural language input, a second signal indicative of a follow-up request to the user interface of the hearing device or the computing device, wherein the follow-up request requests more information related to the issue experienced by the user; and

receive, via the user interface of the hearing device or the computing device, a third signal indicative of a follow-up answer to the follow-up request,

generate, based on the embedding and at least in part based on the follow-up request or the follow-up answer, a first embedding vector;

determine a closest match between the first embedding vector and an embedding vector of the plurality of embedding vectors; and

send, to the user interface of the hearing device or the computing device, a fourth signal indicative of a reply output, the reply output comprising information related to a solution of the related solutions for the issue experienced by the user.

17 . The fitting apparatus for the hearing device of claim 16 , wherein the setting of the hearing device or the computing device comprises at least one of:

a fitting parameter set of a hearing device;

a type of the hearing device;

a coupling setting or type of the hearing device;

a feedback threshold of the hearing device;

a real ear transfer function setting of the hearing device;

a target gain of the hearing device;

actual gain of the hearing device;

an actuator setting of the hearing device;

actual feedback stability of the hearing device; or

a hardware test parameter of the hearing device.

18 . The fitting apparatus for the hearing device of claim 16 , wherein the embedding based on the natural language input comprises embeddings related to multiple dimensions, wherein the first embedding vector is indicative of values associated with each of the embeddings related to multiple dimensions.

19 . The fitting apparatus for the hearing device of claim 16 , wherein the natural language input further comprises temporal information related to the issue experienced by the user, the temporal information indicative of whether the issue is being currently experienced or was previously experienced by the user.

20 . The fitting apparatus for the hearing device of claim 16 , wherein the fitting apparatus is further configured to, upon execution of the instructions:

receive, via the user interface of the hearing device or the computing device, a fifth signal indicative of a confirmatory response to the reply output, the confirmatory response indicative of a user assent to implementing the solution; and

implement the solution in the hearing device or the computing device, or send an instruction to implement the solution to the hearing device or the computing device.