Virtual reading companion technology to support the acquisition of reading fluency
Systems and methods of improving reading fluency by presenting a virtual reading partner that takes turns reading aloud with a reader, are presented. First audio data comprising speech corresponding to the reader reading aloud from a first portion of a written transcript is received. The first audio data is processed to identify spoken words from the speech. The reader's speaking of the words in the first portion of the written transcript are compared to correct speaking of words in the first portion of the written transcript. A reading fluency of the reader is assessed. Second audio data comprising a pre-recorded narration of a second portion of the written transcript or automatically generated text to speech of the second portion is generated. The virtual reading partner is presented to the reader and takes turns reading aloud with the reader until the end of the written transcript has been reached.
1. A computer implemented method of improving reading fluency by presenting a virtual reading partner that takes turns reading aloud with a reader, comprising:
receiving first audio data comprising speech from the reader recorded via a microphone and stored in a computer-readable medium, wherein the speech corresponds to the reader reading aloud from a first portion of a written transcript;
processing the first audio data to identify spoken words from the speech using a processor-implemented automatic speech recognition module;
comparing characteristics of the reader's speaking of words in the first portion of the written transcript to characteristics of correct speaking of the words in the first portion of the written transcript, said characteristics of correct speaking being received over a computer network;
assessing, based on the comparing, a reading fluency of the reader;
generating second audio data comprising (i) a pre-recorded narration of a second portion of the written transcript; or (ii) automatically generated text-to-speech of the second portion comprising a computer-simulated voice synthetization of the second portion, wherein a length of the second portion is based on the reading fluency, wherein the second audio data comprises a computer-simulated segment of the written transcript that is inversely proportional in length with the reading fluency, wherein the length of consecutive computer-simulated segments is adapted during cooperative reading of the written transcript when the reading fluency is detected to have changed during reading of the written transcript, wherein the reading fluency decreases, the length of the adapted consecutive computer-simulated segments increases, and wherein the reading fluency increases, the length of the adapted consecutive-simulated segments decreases; and
presenting to the reader, the virtual reading partner based on audio playback and the adapted consecutive computer-simulated segments of the second audio data, wherein sound recording automatically resumes following presentation of the second audio data, wherein the virtual reading partner reading the second portion and the reader reading the first portion take turns reading aloud based on the length of the adapted consecutive-simulated segments until the end of the written transcript has been reached, wherein the length of the second portion is longer than a length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments increases to a first threshold value, and wherein the length of the second portion is shorter than the length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments decreases to a second threshold value.
2. The method of claim 1 , wherein the comparing further comprises:
comparing the reader's pronunciations of words in the first portion of the written transcript to correct pronunciations of the words in the first portion the written transcript.
3. The method of claim 1 , further comprising:
generating, based on the comparing, a report comprising a measurement at least one of (i) words correct per minute (WCPM); (ii) words correct; (iii) words per minute; (iv) words read; (v) pages read; (vi) time spent reading; (vii) percentage of the chapter read; and (viii) percentage of the book read.
4. The method of claim 3 , wherein the report further comprises at least one of: (i) a list of skipped words; (ii) a list of repeated words; and (iii) a list of incorrectly pronounced words.
5. The method of claim 3 , wherein the report further comprises comprehension data based on at least one of: (i) the reader's ability to read in semantically meaningful chunks; (ii) the reader's ability to correctly pronounce words; (iii) the reader's ability to correctly interpret punctuation; and (iv) the reader's ability to read with expressive interpretation.
6. The method of claim 1 , further comprising:
tracking, based on the comparing, a location in the written transcript that indicates where the reader stopped reading aloud; and
wherein the length of the second portion is further based on the location.
7. The method of claim 3 , wherein the report further comprises engagement data based on at least one of: (i) the reader's facial expressions; (ii) the reader's gross body movements; (iii) the reader's paralinguistic cues; and (iv) the reader's occurrences in audio.
8. A system of improving reading fluency by presenting a virtual reading partner that takes turns reading aloud with a reader, comprising:
one or more data processors;
a non-transitory computer-readable medium encoded with instructions for commanding the one or more data processors to execute steps of a process that include:
receiving first audio data comprising speech from the reader recorded via a microphone and stored in a computer-readable medium, wherein the speech corresponds to the reader reading aloud from a first portion of a written transcript;
processing the first audio data to identify spoken words from the speech using a processor-implemented automatic speech recognition module;
comparing characteristics of the reader's speaking of words in the first portion of the written transcript to characteristics of correct speaking of the words in the first portion of the written transcript, said characteristics of correct speaking being received over a computer network;
assessing, based on the comparing, a reading fluency of the reader;
generating second audio data comprising (i) a pre-recorded narration of a second portion of the written transcript; or (ii) automatically generated text-to-speech of the second portion comprising a computer-simulated voice synthetization of the second portion, wherein a length of the second portion is based on the reading fluency, wherein the second audio data comprises a computer-simulated segment of the written transcript that is inversely proportional in length with the reading fluency, wherein the length of consecutive computer-simulated segments is adapted during cooperative reading of the written transcript when the reading fluency is detected to have changed during reading of the written transcript, and wherein the reading fluency decreases, the length of the adapted consecutive computer-simulated segments increases, wherein the reading fluency increases, the length of the adapted consecutive-simulated segments decreases; and
presenting to the reader, the virtual reading partner based on audio playback and the adapted consecutive computer-simulated segments of the second audio data, wherein sound recording automatically resumes following presentation of the second audio data, wherein the virtual reading partner reading the second portion and the reader reading the first portion take turns reading aloud based on the length of the adapted consecutive-simulated segments until the end of the written transcript has been reached, wherein the length of the second portion is longer than a length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments increases to a first threshold value, and wherein the length of the second portion is shorter than the length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments decreases to a second threshold value.
9. The system of claim 8 , wherein the comparing further comprises:
comparing the reader's pronunciations of words in the first portion of the written transcript to correct pronunciations of the words in first portion the written transcript.
10. The system of claim 8 , wherein the steps further comprise:
generating, based on the comparing, a report comprising a measurement at least one of (i) words correct per minute (WCPM); (ii) words correct; (iii) words per minute; (iv) words read; (v) pages read; (vi) time spent reading; (vii) percentage of the chapter read; and (viii) percentage of the book read.
11. The system of claim 10 , wherein the report further comprises at least one of: (i) a list of skipped words; (ii) a list of repeated words; and (iii) a list of incorrectly pronounced words.
12. The system of claim 10 , wherein the report further comprises comprehension data based on at least one of: (i) the reader's ability to read in semantically meaningful chunks; (ii) the reader's ability to correctly pronounce words; (iii) the reader's ability to correctly interpret punctuation; and (iv) the reader's ability to read with expressive interpretation.
13. The system of claim 8 , wherein the steps further comprise:
tracking, based on the comparing, a location in the written transcript that indicates where the reader stopped reading aloud; and
wherein the length of the second portion is further based on the location.
14. The system of claim 10 , wherein the report further comprises engagement data based on at least one of: (i) the reader's facial expressions; (ii) the reader's gross body movements; (iii) the reader's paralinguistic cues; and (iv) the reader's occurrences in audio.
15. A non-transitory computer-readable medium encoded with instructions for commanding one or more data processors to execute a method of improving reading fluency by presenting a virtual reading partner that takes turns reading aloud with a reader, the method comprising:
receiving first audio data comprising speech from the reader recorded via a microphone and stored in a computer-readable medium, wherein the speech corresponds to the reader reading aloud from a first portion of a written transcript;
processing the first audio data to identify spoken words from the speech using a processor-implemented automatic speech recognition module;
comparing characteristics of the reader's speaking of words in the first portion of the written transcript to characteristics of correct speaking of the words in the first portion of the written transcript, said characteristics of correct speaking being received over a computer network;
assessing, based on the comparing, a reading fluency of the reader;
generating second audio data comprising (i) a pre-recorded narration of a second portion of the written transcript; or (ii) automatically generated text-to-speech of the second portion comprising a computer-simulated voice synthetization of the second portion, wherein a length of the second portion is based on the reading fluency, wherein the second audio data comprises a computer-simulated segment of the written transcript that is inversely proportional in length with the reading fluency, wherein the length of consecutive computer-simulated segments is adapted during cooperative reading of the written transcript when the reading fluency is detected to have changed during reading of the written transcript, and wherein the reading fluency decreases, the length of the adapted consecutive computer-simulated segments increases, wherein the reading fluency increases, the length of the adapted consecutive-simulated segments decreases; and
presenting to the reader, the virtual reading partner based on audio playback and the adapted consecutive computer-simulated segments of the second audio data, wherein sound recording automatically resumes following presentation of the second audio data, wherein the virtual reading partner reading the second portion and the reader reading the first portion take turns reading aloud based on the length of the adapted consecutive-simulated segments until the end of the written transcript has been reached, wherein the length of the second portion is longer than a length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments increases to a first threshold value, and wherein the length of the second portion is shorter than the length of the first portion of the written transcript when the length of the adapted consecutive computer-simulated segments decreases to a second threshold value.
16. The non-transitory computer-readable of claim 15 , wherein the comparing further comprises:
comparing the reader's pronunciations of words in the first portion of the written transcript to correct pronunciations of the words in the first portion the written transcript.
17. The non-transitory computer-readable of claim 15 , the method further comprising:
generating, based on the comparing, a report comprising a measurement at least one of (i) words correct per minute (WCPM); (ii) words correct; (iii) words per minute; (iv) words read; (v) pages read; (vi) time spent reading; (vii) percentage of the chapter read; and (viii) percentage of the book read.
18. The non-transitory computer-readable of claim 17 , wherein the report further comprises at least one of: (i) a list of skipped words; (ii) a list of repeated words; and (iii) a list of incorrectly pronounced words.
19. The non-transitory computer-readable of claim 17 , wherein the report further comprises comprehension data based on at least one of: (i) the reader's ability to read in semantically meaningful chunks; (ii) the reader's ability to correctly pronounce words; (iii) the reader's ability to correctly interpret punctuation; and (iv) the reader's ability to read with expressive interpretation.
20. The non-transitory computer-readable of claim 15 , the method further comprising:
tracking, based on the comparing, a location in the written transcript that indicates where the reader stopped reading aloud; and
wherein the length of the second portion is further based on the location.