Speech detection from facial skin movements
Systems and methods are disclosed for speech detection from facial skin movements. In one implementation, a system may include at least one light source, at least one sensor configured to receive light reflections from the at least one light source; and a processor configured to control the at least one light source to illuminate a region of a face of a user in a time interval. The processor may receive reflection signals indicative of light reflected from the face in the time interval. The reflection signals may be analyzed to determine facial skin movements in the time interval, wherein determining the sequence of words includes using an artificial neural network and a motion-to-language analysis. Then, based on the determined facial skin movements, the processor may determine a sequence of words associated with the facial skin movements, and output a textual transcription corresponding with the determined sequence of words.
1. A light sensing system for determining textual transcription from minute facial skin movements, the system comprising:
at least one light source;
at least one sensor configured to receive light reflections from the at least one light source;
at least one processor configured to:
control the at least one light source to illuminate a region of a face of a user;
receive from the at least one sensor, reflection signals indicative of light reflected from the face in a time interval;
analyze the reflection signals to determine minute facial skin movements in the time interval;
based on the determined minute facial skin movements in the time interval, determine a sequence of words associated with the minute facial skin movements, wherein determining the sequence of words includes using an artificial neural network and a motion-to-language analysis; and
output a textual transcription corresponding with the determined sequence of words.
2. The light sensing system of claim 1 , wherein the minute facial skin movements are associated with a vocalization of the sequence of words in a first language and the wherein the at least one processor is further configured to translate the textual transcription to a language other than the first language.
3. The light sensing system of claim 1 , wherein the at least one processor is further configured to generate speech output based on the facial skin movements.
4. The light sensing system of claim 1 , wherein the at least one processor is further configured to generate an email draft from the textual transcription.
5. The light sensing system of claim 1 , wherein the at least one processor is further configured to generate a synthesized audio signal based on the facial skin movements.
6. The light sensing system of claim 1 , wherein analyzing the reflection signals to determine minute facial skin movements includes detection of changes in speckle patterns that occur due to the minute facial skin movements.
7. The light sensing system of claim 1 , wherein the reflection signals were during a conversation and determining the sequence of words involves determining a context of the conversation.
8. The light sensing system of claim 1 , wherein the system is associated with a microphone.
9. A light sensing system for determining speech based on minute facial skin movements, the sensing system comprising:
a housing configured to be worn on a head of a user and to be supported by an ear of the user;
at least one light source associated with the housing and configured to direct light towards a facial region of the head;
at least one sensor associated with the housing and configured to receive light source reflections from the facial region and to output associated reflection signals;
at least one processor configured to:
receive the reflection signals from the at least one sensor in a time interval;
analyze the reflection signals to identify minute facial movements in the time interval, wherein analyzing the reflection signals to identify the minute facial skin movements includes detecting changes in speckle patterns that occur due to the minute facial skin movements;
decipher the minute facial movements to determine associated speech; and
generate output of the associated speech.
10. The light sensing system of claim 9 , wherein the housing is configured, when worn, to assume an aiming direction of the at least one light source for illuminating a portion of a cheek of the user.
11. The light sensing system of claim 9 , wherein the housing is associated with an earphone.
12. The light sensing system of claim 11 , wherein the earphone has an arm extending therefrom and wherein the at least one light source is located in the arm.
13. The light sensing system of claim 9 , further comprising a microphone configured to sense sounds uttered by the user.
14. The light sensing system of claim 13 , wherein the at least one processor is configured to calibrate spoken words detected by the microphone with the identified minute facial movements.
15. A light sensing system for determining silent speech based on minute facial skin movements, the light sensing system, comprising:
at least one wearable light source configured to direct light towards a portion of a cheek of a user;
at least one wearable sensor configured to receive light source reflections from the portion of the cheek and to output associated reflection signals;
at least one processor operable in an idle mode and in a high power mode;
wherein in the idle mode, the at least one processor is configured to receive the reflection signals from the at least one sensor, process the reflection signals to identify from reflection signals associated at least one movement in the portion of the cheek at least one trigger, and automatically switch to the high power mode upon identification of the at least one trigger;
wherein in the high power mode, the at least one processor is configured to analyze the reflection signals from the portion of the cheek to identify facial movements associated with silent speech; and
wherein following the identification of the facial movements associated with the silent speech, the at least one processor is configured to decipher the facial movements in the portion of the cheek and generate an output associated with the silent speech.
16. The light sensing system of claim 15 , wherein the facial movements associated with silent speech include minute movements manifest in cheek skin of the portion of the cheek, and wherein the at least one processor is configured to generate the output associated with the silent speech by deciphering the movements manifest in the cheek skin.
17. The light sensing system of claim 15 , wherein the system is associated with a microphone.
18. The light sensing system of claim 15 , wherein, wherein a frame rate in the idle mode is lower than a frame rate in the high power mode.
19. The light sensing system of claim 15 , wherein the at least one processor is further configured to determine a sequence of words from the identified deciphered facial movements.
20. The light sensing system of claim 19 , wherein determining the sequence of words involves using a neural network associated with personal training data.