Determination of transcription accuracy
A method may include obtaining audio of a communication session between a first device of a first user and a second device of a second user. The method may further include obtaining a transcription of second speech of the second user. The method may also include identifying one or more first sound characteristics of first speech of the first user. The method may also include identifying one or more first words indicating a lack of understanding in the first speech. The method may further include determining an experienced emotion of the first user based on the one or more first sound characteristics. The method may also include determining an accuracy of the transcription of the second speech based on the experienced emotion and the one or more first words.
1. A method comprising: obtaining audio of a communication session between a first device of a first user and a second device of a second user, the communication session configured for verbal communication such that the audio includes first speech of the first user and second speech of the second user;
obtaining a transcription of the second speech of the second user;
identifying one or more first sound characteristics of the first speech of the first user;
identifying one or more first words in the transcription indicating the first user lacks understanding of the second speech;
based on the one or more first sound characteristics, determining an experienced emotion of the first user; and
based on the experienced emotion and the one or more first words, determining an accuracy of the transcription of the second speech.
2. The method of claim 1 , wherein the first sound characteristics of the first speech include: a tone, a volume, a pitch, an inflection, a timbre, or a speed of the first speech.
3. The method of claim 1 , further comprising:
obtaining a second transcription of the first speech of the first user; and
identifying one or more second words in the second transcription indicating the first user lacks understanding of the second speech,
wherein the accuracy of the transcription of the second speech is determined further based on the second words.
4. The method of claim 1 , further comprising:
identifying one or more second sound characteristics of the second speech,
wherein determining the accuracy of the transcription of the second speech is further based on the one or more second sound characteristics.
5. The method of claim 1 , further comprising:
obtaining image data of the communication session between the first device of the first user and the second device of the second user, the image data including images of the first user; and
identifying one or more facial expressions of the first user in the image data,
wherein determining the experienced emotion of the first user is further based on the one or more facial expressions.
6. The method of claim 5 , further comprising:
determining, based on the image data, the first user is not viewing the transcription during a first duration of time; and
determining, based on the image data, the first user is viewing the transcription during a second duration of time,
wherein determining the accuracy of the transcription of the second speech comprises determining the accuracy of the transcription during the second duration of time but not during the first duration of time.
7. The method of claim 1 , further comprising:
determining a first topic of the communication session;
based on the first topic, identifying an expected emotion for the first user; and
based on the expected emotion and the experienced emotion, determining an unexpected emotion of the first user,
wherein determining the accuracy of the transcription of the second speech is further based on the unexpected emotion.
8. A method comprising:
obtaining audio of a communication session between a first device of a first user and a second device of a second user, the communication session configured for verbal communication such that the audio includes first speech of the first user;
obtaining a transcription of the communication session;
determining a first topic of the communication session;
based on the first topic, identifying an expected emotion for the first user;
obtaining one or more first sound characteristics of the first speech;
based on the one or more first sound characteristics, determining an experienced emotion of the first user;
based on the expected emotion and the experienced emotion, determining an unexpected emotion of the first user; and
based on the unexpected emotion, determining an accuracy of the transcription.
9. The method of claim 8 , wherein the first sound characteristics of the first speech include: a tone, a volume, a pitch, an inflection, a timbre, or a speed of the first speech.
10. The method of claim 8 , wherein the audio further includes second speech of the second user and the method further comprises:
identifying one or more first words indicating a lack of understanding in the first speech; identifying one or more second sound characteristics of the second speech; and
identifying one or more repeated words in the second speech,
wherein determining the accuracy of the transcription is further based on the one or more first words, the one or more second sound characteristics, and the one or more repeated words.
11. The method of claim 10 , wherein the first words indicating a lack of understanding include: requests for the second user to repeat one or more portions of the second speech, question words, and apologies.
12. The method of claim 10 , wherein the one or more second sound characteristics of the second speech include: an increasing volume or a decreasing speed of the second speech.
13. The method of claim 8 , further comprising:
obtaining image data of the communication session between the first device of the first user and the second device of the second user, the image data including images of the first user; and
identifying one or more facial expressions of the first user in the image data,
wherein determining the experienced emotion of the first user is further based on the one or more facial expressions.
14. A system comprising: one or more processors; and one or more non-transitory computer-readable media configured to store instructions that in response to being executed by the one or more processors cause the system to perform operations, the operations comprising:
obtaining audio of a communication session between a first device of a first user and a second device of a second user, the communication session configured for verbal communication such that the audio includes first speech of the first user and second speech of the second user;
obtaining a transcription of the second speech of the second user;
identifying one or more first sound characteristics of the first speech of the first user;
identifying one or more first words in the transcription indicating the first user lacks understanding of the second speech;
based on the one or more first sound characteristics, determining an experienced emotion of the first user; and
based on the experienced emotion and the one or more first words, determining an accuracy of the transcription of the second speech.
15. The system of claim 14 , wherein the first sound characteristics of the first speech include: a tone, a volume, a pitch, a timbre, and a speed of the first speech.
16. The system of claim 14 , wherein the operations further comprise:
obtaining a second transcription of the first speech of the first user; and
identifying one or more second words in the second transcription indicating the first user lacks understanding of the second speech,
wherein the accuracy of the transcription of the second speech is determined further based on the second words.
17. The system of claim 14 , wherein the operations further comprise:
identifying one or more second sound characteristics of the second speech; and
wherein determining the accuracy of the transcription of the second speech is further based on the one or more second sound characteristics.
18. The system of claim 17 , wherein the one or more second sound characteristics of the second speech include: an increasing volume of the second speech or a decreasing speed of the second speech.
19. The system of claim 14 , wherein the operations further comprise: obtaining image data of the communication session between the first device of the first user and the second device of the second user, the image data including images of the first user; and
identifying one or more facial expressions of the first user in the image data,
wherein determining the experienced emotion of the first user is further based on the one or more facial expressions.
20. The system of claim 14 , wherein the operations further comprise: determining a first topic of the communication session;
based on the first topic, identifying an expected emotion for the first user; and
based on the expected emotion and the experienced emotion, determining an unexpected emotion of the first user,
wherein determining the accuracy of the transcription of the second speech is further based on the unexpected emotion.