Eyewear including diarization
An eyewear device that performs diarization by segmenting spoken language into different speakers and remembering each speaker over the course of a session. The speech of each speaker is translated to text and the text of each speaker is displayed on an eyewear display. The text of each user has a different attribute such that the eyewear user can distinguish the text of different speakers. Examples of the text attribute can be a text color, font, and font size. The text is displayed on the eyewear display such that it does not substantially obstruct the user's vision.
1. Eyewear, comprising:
a frame;
a display supported by the frame;
a microphone coupled to the frame; and
a camera configured to generate an image including an object; and
an electronic processor configured to:
receive speech from a plurality of human speakers via the microphone;
identify the plurality of human speakers;
perform diarization on the received speech to segment spoken language into different speakers;
display text associated with each speaker on the display;
display a user created graphical depiction of a person associated with and indicative of the identified speaker proximate the text of the associated speaker such that an eyewear user can visually associate the text to the respective speaker;
process pitch and intonation of the received speech;
establish a color for received speech based on the pitch and intonation;
display the text in the established color based on the pitch and intonation;
adjust font size of the text by increasing a font attribute based on a decibel level of the received speech above a first threshold and decreasing the font attribute based on a decibel level of the received speech below a second threshold;
determine the object in the image; and
generate speech indicative of the object responsive to a speech command.
2. The eyewear of claim 1 , wherein the processor is configured to use a convolutional neural network (CNN) to perform the diarization.
3. The eyewear of claim 2 , wherein the text of each speaker has a unique color.
4. The eyewear of claim 2 , wherein the text of each speaker has a unique font size.
5. The eyewear of claim 2 , wherein the text of each speaker has a unique font style.
6. A method for use with eyewear, the eyewear having a frame, a display supported by the frame, a microphone coupled to the frame, a camera configured to generate an image including an object, and an electronic processor, the processor:
receiving speech from a plurality of human speakers via the microphone;
identifying the plurality of human speakers;
performing diarization on the received speech to segment spoken language into different speakers;
displaying text associated with each speaker on the display; and
displaying a user created graphical depiction of a person associated with and indicative of the identified speaker proximate the text of the associated speaker such that an eyewear user can visually associate the text to the respective speaker;
processing pitch and intonation of the received speech;
establishing a color for received speech based on the pitch and intonation;
displaying the text in the established color based on the pitch and intonation;
adjusting font size of the text by increasing a font attribute based on a decibel level of the received speech above a first threshold and decreasing the font attribute based on a decibel level of the received speech below a second threshold;
determining the object in the image; and
generating speech indicative of the object responsive to a speech command.
7. The method of claim 6 , wherein the processor uses a convolutional neural network (CNN) to perform the diarization.
8. The method of claim 7 , wherein the text of each speaker has a unique color.
9. The method of claim 7 , wherein the text of each speaker has a unique font size.
10. The method of claim 7 , wherein the text of each speaker has a unique font style.
11. A non-transitory computer readable medium storing program code which, when executed by a processor of eyewear having a frame, a display supported by the frame, a microphone coupled to the frame, a camera configured to generate an image including an object, is operative to cause the processor to perform the steps of:
receiving speech from a plurality of human speakers via the microphone;
identifying the plurality of human speakers;
performing diarization on the received speech to segment spoken language into different speakers;
displaying text associated with each speaker on the display; and
displaying a user created graphical depiction of a person associated with and indicative of the identified speaker proximate the text of the associated speaker such that an eyewear user can visually associate the text to the respective speaker;
processing pitch and intonation of the received speech;
establishing a color for received speech based on the text, pitch; and intonation;
displaying the text in the established color based on the text, pitch, and intonation;
adjusting font size of the text by increasing a font attribute based on a decibel level of the received speech above a first threshold and decreasing the font attribute based on a decibel level of the received speech below a second threshold;
determining the object in the image; and
generating speech indicative of the object responsive to a speech command.
12. The non-transitory computer readable medium as specified in claim 11 , wherein the program code, when executed, is operative to cause the processor to use a convolutional neural network (CNN) to perform the diarization.
13. The non-transitory computer readable medium as specified in claim 12 , wherein the text of each speaker has a unique color.
14. The non-transitory computer readable medium as specified in claim 12 , wherein the text of each speaker has a unique font size or a font type.