Sensor based text-to-speech emotional conveyance
A computer-implemented method includes determining, by a first device, a current emotional state of a user of the first device. The current emotional state is based, at least in part, on real-time information corresponding to the user and relates to a textual message from the user. The computer-implemented method further includes determining, by the first device, a set of phonetic data associated with a plurality of vocal samples corresponding to the user. The computer-implemented method further includes dynamically converting, by the first device, the textual message into an audio message. The audio message is converted from the textual message into the audio message based, at least in part, on the current emotional state and a portion of the set of phonetic data that corresponds to the current emotional state. A corresponding computer system and computer program product are also disclosed.
1. A method for assigning phonetic data to a textual message for synthesis of an audio version of the textual message, the method comprising the steps of:
receiving, from a user of a mobile device, a plurality of vocal samples;
determining a first set of phonetic data from the plurality of vocal samples based, at least in part, on:
segmenting each vocal sample in the plurality of vocal samples into a plurality of phonetic units; and
identifying a plurality of acoustic parameters corresponding to each of the plurality of phonetic units;
indexing a first set of phonetic units derived from the plurality of phonetic units based, at least in part, on:
identifying a correlation between acoustic parameters corresponding to phonetic units segmented from the plurality of vocal samples and a first emotional state of the user during a period of time in which each of the vocal samples are generated;
determining, based on physical characteristics associated with the user of the mobile device, the first emotional state of the user during a period of time which the textual message is generated via the mobile device;
comparing the first emotional state of the user with a context of the textual message;
determining, based on the comparison, that the first emotional state of the user matches the context of the textual message;
assigning, in response to determining that the first emotional state of the user matches the context of the textual message, acoustic parameters associated with those phonetic units in the first set of phonetic units that correspond to phonetic transcriptions of the textual message, wherein: the acoustic parameters are predicted to convey, to a listener, the first emotional state of the user; and
synthesizing, via speech synthesizer software, an audio version of the textual message based, at least in part, on the acoustic parameters.
2. The method of claim 1 , wherein determining the first emotional state of the user is further based on:
comparing baseline physical characteristics corresponding to the user with real-time physical characteristics corresponding to the user.
3. The method of claim 1 , further comprising:
comparing the emotional context of the textual message with the first emotional state of the user; and
determining that one or more words associated with the textual message do not correspond to the first emotional state of the user.
4. The method of claim 3 , wherein:
responsive to determining that the one or more words do not correspond to the first emotional state of the user, flagging the one or more words that do not correspond to the first emotional state of the user.
5. The computer method of claim 4 , further comprising:
suggesting one or more replacement words for the one or more words that are flagged.