Earbud with facial micromovement detection capabilities
A multifunctional earpiece comprising an ear-mountable housing, a speaker integrated with the ear-mountable housing for presenting sound, a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face, a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, and wherein the multifunctional earpiece is configured to simultaneously present the sound through the speaker, project the light toward the skin, and detect the received reflections indicative of the prevocalized words.
1. A multifunctional earpiece, comprising:
an ear-mountable housing;
a speaker integrated with the ear-mountable housing for presenting sound;
a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face;
a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and
wherein the multifunctional earpiece is configured to simultaneously present the sound through the speaker, project the light toward the skin, and use image processing on the produced at least one image to determine the prevocalized words.
2. The multifunctional earpiece of claim 1 , wherein at least a portion of the ear-mountable housing is configured to be placed in an ear canal.
3. The multifunctional earpiece of claim 1 , wherein at least a portion of the ear-mountable housing is configured to be placed over or behind an ear.
4. The multifunctional earpiece of claim 1 , further comprising at least one processor configured to output via the speaker an audible simulation of the prevocalized words determined from at least one image.
5. The multifunctional earpiece of claim 4 , wherein the audible simulation of the prevocalized words includes a synthetization of a voice of an individual other than the wearer.
6. The multifunctional earpiece of claim 4 , wherein the audible simulation of the prevocalized words includes a synthetization of the prevocalized words in a first language other than a second language of the prevocalized words.
7. The multifunctional earpiece of claim 1 , further comprising a microphone integrated with the ear-mountable housing for receiving audio indicative of a wearer's speech.
8. The multifunctional earpiece of claim 1 , wherein the light source is configured to project a pattern of coherent light toward the skin of the wearer's face, the pattern including a plurality of spots.
9. The multifunctional earpiece of claim 1 , wherein the light detector is configured to output associated reflection signals indicative of muscle fiber recruitments.
10. The multifunctional earpiece of claim 9 , wherein the recruited muscle fibers include at least one of zygomaticus muscle fibers, orbicularis oris muscle fibers, risorius muscle fibers, or levator labii superioris alaeque nasi muscle fibers.
11. The multifunctional earpiece of claim 1 , further comprising at least one processor configured to analyze the light reflections to determine the facial skin micromovements.
12. The multifunctional earpiece of claim 11 , wherein the analysis includes speckle analysis.
13. The multifunctional earpiece of claim 11 , further comprising a microphone integrated with the ear-mountable housing for receiving audio indicative of a wearer's speech, and wherein the at least one processor is configured to use the audio received via the microphone and the reflections received via the light detector to correlate facial skin micromovements with spoken words and to train a neural network to determine subsequent prevocalized words from subsequent facial skin micromovements.
14. The multifunctional earpiece of claim 11 , wherein the at least one processor is configured to identify a trigger in the determined facial skin micromovements for activating the microphone.
15. The multifunctional earpiece of claim 11 , further comprising a pairing interface for pairing with a communications device, and wherein the at least one processor is configured to transmit an audible simulation of the prevocalized words to the communications device.
16. The multifunctional earpiece of claim 11 , further comprising a pairing interface for pairing with a communications device, and wherein the at least one processor is configured to transmit a textual presentation of the prevocalized words to the communications device.
17. The multifunctional earpiece of claim 1 , wherein the light source is configured to project coherent light toward the skin of the wearer's face.
18. The multifunctional earpiece of claim 1 , wherein the light source is configured to project a noncoherent light toward the skin of the wearer's face.
19. A method for operating a multifunctional earpiece, the method comprising:
operating a speaker integrated with an ear-mountable housing associated with the multifunctional earpiece for presenting sound;
operating a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face;
operating a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and
simultaneously presenting the sound through the speaker, projecting the light toward the skin, and using image processing on the produced at least one image to determine the prevocalized words.
20. A non-transitory computer readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for operating a multifunctional earpiece, the operations comprising:
operating a speaker integrated with an ear-mountable housing associated with the multifunctional earpiece for presenting sound;
operating a light source integrated with the ear-mountable housing for projecting light toward skin of the wearer's face;
operating a light detector integrated with the ear-mountable housing and configured to receive reflections from the skin corresponding to facial skin micromovements indicative of prevocalized words of the wearer, the light detector including one or more multi-pixel sensors for enabling production of at least one image providing spatial information beyond a single point; and
simultaneously presenting the sound through the speaker, projecting the light toward the skin, and using image processing on the produced at least one image to determine the prevocalized words.