Hearing aid control method, and hearing aid device and storage medium

1 . A hearing aid control method, comprising:

providing a hearing aid device, wherein the hearing aid device comprises a pair of Augmented Reality (AR) glasses, a bone conduction vibration sensor in the pair of AR glasses, a sound collection device, and an in-ear broadcasting device that are provided in the pair of AR glasses, wherein the sound collection device is configured to collect speech, the in-ear broadcasting device is configured to play audio, and the bone conduction vibration sensor is touchable to a skull region of the user;

displaying a hearing detection image in a viewing window area of the pair of AR glasses, playing hearing test audio to acquire a feedback signal from a user wearing the hearing aid device based on the hearing test audio, and determining a hearing assessment result for the user based on the feedback signal, wherein the hearing assessment result comprises a hearing level;

in response to determining, based on the hearing assessment result, that a display operation is required to be performed by the hearing aid device, collecting speech information; and

converting the speech information into text information upon the collection of the speech information, and displaying the text information in the viewing window area of the pair of AR glasses, by:

detecting, based on the bone conduction vibration sensor, a vibration signal of a vocal cord of the user upon collecting the speech information;

detecting the vibration signal of the vocal cord based on a feature comparison model and determining, based on a detection result, whether the user is a sound source of the speech information; and

determining, based on a result of the determination whether the user is the sound source of the speech information, a corresponding rendering method for displaying the text information in the viewing window area of the pair of AR glasses, comprising:

performing the display operation based on a first rendering method in response to determining that the user is not the sound source of the speech information; and

performing the display operation based on a second rendering method in response to determining that the user is the sound source of the speech information when converting the speech information into the text information;

wherein the first rendering method and the second rendering method are different rendering methods configured based on at least one of: color, font, display ratio, or display speed; and

when displaying the text information based on the second rendering method, receiving feedback information from the user to determine a pronunciation level of the user based on the feedback information.

2 . The method of claim 1 , wherein converting the speech information into the text information, and displaying the text information in the viewing window area of the pair of AR glasses, further comprises:

determining a sound source orientation of the speech information;

performing face recognition on an image within a range of a viewing window of the pair of the AR glasses based on the sound source orientation to identify a speaking subject of the speech information; and

converting the speech information into the text information and displaying the text information in the viewing window area, corresponding to the speaking subject, on the hearing aid device.

3 . The method of claim 1 , wherein converting the speech information into the text information, and displaying the text information in the viewing window area of the pair of AR glasses, further comprises:

detecting a spectral parameter of the speech information;

distinguishing a gender of a sound source of the speech information based on the spectral parameter;

determining a corresponding rendering method based on the gender of the sound source; and

determining a display style of the text information based on the corresponding rendering method and displaying the text information in the viewing window area.

4 . The method of claim 2 , wherein converting the speech information into the text information, and displaying the text information in the viewing window area of the pair of AR glasses, further comprises:

detecting a distance between the user and the speaking subject based on a visual feature of the speaking subject; and

synchronously adjusting a size of a text box of the text information based on the detected distance between the user and the speaking object.

5 . The method of claim 1 , wherein converting the speech information into the text information upon the collection of the speech information, and displaying the text information in the viewing window area of the pair of AR glasses, further comprises:

invoking, in response to detecting that the collected speech information is to-be-translated speech information, a translation model of a target language to translate the to-be-translated speech information, and obtaining translated text; and

displaying the translated text as the text information in the viewing window area of the pair of AR glasses.

6 . The method of claim 1 , wherein the hearing detection image comprises a plurality of groups of different combinations of long and short graphics, and characters corresponding to each group of combinations of long and short graphics; and

wherein playing the hearing test audio to acquire the feedback signal from the user wearing the hearing aid device based on the hearing test audio, and determining the hearing assessment result for the user based on the feedback signal, further comprises:

playing, during an assessment, at least one of a plurality of groups of sounds with long and short lengths based on a specified sound volume and/or sound pitch as the hearing detection audio, each group of sounds with long and short lengths corresponding to a group of combinations of long and short graphics;

receiving a feedback result from the user on an identification of the sounds with long and short lengths as the feedback signal; and

determining the hearing assessment result for the user based on the feedback result, wherein the sound volume comprises bass, alto, and soprano, and the sound pitch comprises low frequency, mid-frequency, and high frequency, and wherein the method further comprises performing a plurality of assessments based on different sound volumes and/or different sound pitches.

7 . The method of claim 6 , wherein receiving the feedback result from the user on the identification of the sounds with long and short lengths, further comprises:

displaying, after playing the sounds with long and short lengths, a correct option and an incorrect option for the characters corresponding to the sounds with long and short lengths in the viewing window; and

receiving a selection result from the user for the correct option and the incorrect option, and determining the selection result as the feedback result.

8 . The method of claim 6 , wherein receiving the feedback result from the user on the identification of the sounds with long and short lengths, further comprises:

collecting a recognition speech from the user for the characters corresponding to the sounds with long and short lengths, and determining the recognition speech as the feedback result.

9 . The method of claim 6 , wherein determining the hearing assessment result for the user based on the feedback result, further comprises:

determining, based on the feedback result, a feedback character fed back from the user;

detecting whether the feedback character is correct; and

determining, based on the detection result, a volume range recognizable to the user and a type of pitch loss of the user, as the hearing assessment result for the user.

10 . The method of claim 9 , wherein, before collecting the speech information for auxiliary processing in response to determining, based on the hearing assessment result, that the display operation is required to be performed by the hearing aid device, the method further comprises:

in response to determining that the volume range is a first volume range, performing an amplification operation on the collected speech information;

in response to determining that the volume range is a second volume range, performing the amplification operation and the display operation on the collected speech information; and

in response to determining that the volume range is a third volume range, performing the display operation on the collected speech information.

11 . The method of claim 10 , wherein performing the amplification operation on the collected speech information, further comprises:

detecting an intensity parameter and a frequency parameter of the speech information; and

adopting a dynamic amplifier to automatically adjust a gain of the intensity parameter and the frequency parameter to adjust the intensity parameter and the frequency parameter to a comfortable listening range.

12 . The method of claim 10 , wherein performing the amplification operation on the collected speech information, further comprises:

upon detecting a presence of the pitch loss of the user, performing a compensation operation, according to the type of the pitch loss of the user, for missing frequencies of the speech information.

13 . A hearing aid device, comprising:

a pair of Augmented Reality (AR) glasses;

a bone conduction vibration sensor provided in the pair of AR glasses, the bone conduction vibration sensor being touchable to a region of a vocal cord of the user and configured to detect a vibration signal of the vocal cord of the user;

an in-ear broadcasting device provided in the pair of AR glasses, configured to play hearing test audio;

a processor implemented in hardware configured to acquire a feedback signal from a user wearing the hearing aid device based on the hearing test audio, and to determine a hearing assessment result for the user based on the feedback signal; and

a sound collection device provided in the pair of AR glasses, configured to collect speech information, in response to determining, based on the hearing assessment result, that a display operation is required to be performed by the hearing aid device;

wherein the processor is further configured to convert the speech information into text information upon the collection of the speech information,

wherein the processor is further configured to detect the vibration signal of the vocal cord based on a feature comparison model to determine whether the user is sound source of the speech information; and

wherein the pair of AR glasses is further configured to: perform the display operation based on a first rendering method in response to determining that the user is not the sound source of the speech information; and perform the display operation based on a second rendering method in response to determining that the user is the sound source of the speech information.

14 . A hearing aid device, comprising:

a pair of augmented reality (AR) glasses, a bone conduction vibration sensor in the pair of AR glasses, a sound collection device, and an in-ear broadcasting device, the sound collection device being configured to collect speech, the bone conduction vibration sensor being touchable to a skull region of the user, and the in-ear broadcasting device being configured to play audio;

at least one hardware processor; and

program instructions stored in memory that are executable by the at least one hardware processor that, when executed, direct the at least one hardware processor to:

display a hearing detection image in a viewing window area of the pair of AR glasses, playing hearing test audio to acquire a feedback signal from a user wearing the hearing aid device based on the hearing test audio, and determining a hearing assessment result for the user based on the feedback signal, wherein the hearing assessment result comprises a hearing level;

in response to determining, based on the hearing assessment result, that a display operation is required to be performed by the hearing aid device, collect speech information; and

convert the speech information into text information upon the collection of the speech information, and display the text information in the viewing window area of the pair of AR glasses by:

detecting a vibration signal of a vocal cord of the user based on the bone conduction vibration sensor upon the collection of the speech information; detecting the vibration signal of the vocal cord based on a feature comparison model and determining whether the user is sound source of the speech information based on a detection result; and determining, based on a result of the determining whether the user is the sound source of the speech information, a corresponding rendering method for displaying, based on the corresponding rendering method, the text information in the viewing window area of the pair of AR glasses, comprising:

performing the display operation based on a first rendering method in response to determining that the user is not the sound source of the speech information; and performing the display operation based on a second rendering method in response to determining that the user is the sound source of the speech information when converting the speech information into the text information;

wherein the first rendering method and the second rendering method are different rendering methods configured based on at least one of: color, font, display ratio, or display speed; and

when displaying the text information based on the second rendering method, receive feedback information from the user to determine a pronunciation level of the user based on the feedback information.

15 . A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a hardware processor, causes the hardware processor to implement the method of claim 1 .