IP Library Granted Patent US 10,999,415
Granted Patent B2
US 10,999,415 · App. 16/195,790 · Granted May 4, 2021

Creating a cinematic storytelling experience using network-addressable devices

Inventors: Alexander Rothera (Chicago, IL); Scott D. Lange (Birmingham, MI); Jennifer Drop (Brooklyn, NY)
Assignee: Google LLC
H04L67/42G06F40/00G10L15/00H04L67/10G10L15/26H04L67/306H04M3/4938H04M2201/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,999,415
App. No.
16/195,790
Granted
May 4, 2021
Kind
B2
Abstract

A computer-implemented method to create a cinematic storytelling experience is provided. The method includes receiving, from a client device, a user selection of a story, and obtaining, from a server, (i) a lexicon associated with the selected story, the lexicon comprising a limited number of keywords, and (ii) instructions associated with the selected story, the instructions including instructions to perform at least one action responsive to the client device detecting a keyword. The method further includes receiving user input to initiate verbal recitation of the selected story, detecting correspondence between the verbal recitation and at least one of the keywords, and performing the at least one action according to the instructions for the at least one keyword.

Claims (44)

1. A computer-implemented method comprising:

receiving, via a user interface on a client device, a user selection of a story to be recited as a verbal narration of a book with sound effects providing animation of the selected story;

obtaining, from a server, (i) a lexicon associated with the selected story, the lexicon comprising a limited number of keywords and data indicating a location of each keyword within the selected story, and (ii) information identifying actions associated with the selected story, wherein at least one action of the identified actions is to play, responsive to the client device detecting at least one keyword of the limited number of keywords during a verbal recitation of the selected story, audio content with a sound effect that animates an event described by the at least one keyword;

receiving user input to initiate verbal recitation of the selected story;

detecting the at least one keyword during the verbal recitation of the selected story, wherein each of the at least one keyword is detected based on the lexicon associated with the selected story, respective one or more words from the selected story that are being verbally recited, and a location of the respective one or more words within the selected story; and

performing, by a processing device, the at least one action responsive to detecting the at least one keyword during the verbal recitation of the selected story, wherein the at least one action comprises playing the audio content with the sound effect that animates the event described by the at least one keyword in the selected story.

2. The computer-implemented method of claim 1 , wherein the client device is a voice activated home assistant device.

3. The computer-implemented method of claim 1 , wherein the lexicon further comprises a total number of words in the selected story, wherein data indicating the location of each keyword within the selected story is represented by a word count associated with a respective keyword in view of the total number of words, and wherein detecting the at least one keyword further comprises:

sending data defining the verbal recitation of the selected story to the server, the sent data comprising locations of respective verbally recited words within the selected story, wherein each location is represented by a corresponding word count; and

receiving from the server correspondence between the data defining the verbal recitation of the selected story and the at least one keyword.

4. The computer-implemented method of claim 1 , wherein the detecting comprises using a matching algorithm to match an audio profile of the at least one keyword with an audio profile of the user input.

5. The computer-implemented method of claim 1 , wherein the verbal recitation comprises playback of a recorded narration of the story, the recorded narration received from the server in response to the user selection of the story.

6. The computer-implemented method of claim 1 , wherein the verbal recitation is a narration of the selected story by a user of the client device, and wherein the narration of the selected story is received via a microphone of the client device.

7. The computer-implemented method of claim 1 , wherein the at least one action is performed at a plurality of points in the story, and wherein a sound characteristic of the sound effect provided by the at least one action changes across the plurality of points in the story.

8. A computer system, comprising:

a memory; and

a processing device, coupled to the memory, to:

receive, via a user interface on a client device, a user selection of a story to be recited as a verbal narration of a book with sound effects providing animation of the selected story;

obtain, from a server, (i) a lexicon associated with the selected story, the lexicon comprising a limited number of keywords and data indicating a location of each keyword within the selected story, and (ii) information identifying actions associated with the selected story, wherein at least one action of the identified actions is to play, responsive to the client device detecting at least one keyword of the limited number of keywords during a verbal recitation of the selected story, audio content with a sound effect that animates an event described by the at least one keyword;

receive user input to initiate verbal recitation of the selected story;

detect the at least one keyword during the verbal recitation of the selected story, wherein each of the at least one keyword is detected based on the lexicon associated with the selected story, respective one or more words from the selected story that are being verbally recited, and a location of the respective one or more words within the selected story; and

perform the at least one action responsive to detecting the at least one keyword during the verbal recitation of the selected story, wherein the at least one action comprises playing the audio content with the sound effects that animates the event described by the at least one keyword in the selected story.

9. The system of claim 8 , wherein the client device is a voice activated home assistant device.

10. The system of claim 8 , wherein the lexicon further comprises a total number of words in the selected story, wherein data indicating the location of each keyword within the selected story is represented by a word count associated with a respective keyword in view of the total number of words, and wherein to detect the at least one keyword, the processing device is further to:

send data defining the verbal recitation of the selected story to the server, the sent data comprising locations of respective verbally recited words within the selected story, wherein each location is represented by a corresponding word count; and

receive from the server correspondence between the data defining the verbal recitation of the selected story and the at least one keyword.

11. The system of claim 8 , wherein to detect the at least one keyword, the processing device is further to:

use a matching algorithm to match an audio profile of the at least one keyword with an audio profile of the user input.

12. The system of claim 8 , wherein the verbal recitation comprises playback of a recorded narration of the story, the recorded narration received from the server in response to the user selection of the story.

13. The system of claim 8 , wherein the verbal recitation is a narration of the selected story by a user of the client device, and wherein the narration of the selected story is received via a microphone of the client device.

14. The system of claim 8 , wherein the at least one action is performed at a plurality of points in the story, and wherein a sound characteristic of the sound effect provided by the at least one action changes across the plurality of points in the story.

15. A non-transitory computer-readable medium storing computer-executable instructions that, in response to execution, cause a processing device to perform operations comprising:

receiving, via a user interface on a client device, a user selection of a story to be recited as a verbal narration of a book with sound effects providing animation of the selected story;

obtaining, from a server, (i) a lexicon associated with the selected story, the lexicon comprising a limited number of keywords and data indicating a location of each keyword within the selected story, and (ii) information identifying actions associated with the selected story, wherein at least one action of the identified actions is to play, responsive to the client device detecting at least one keyword of the limited number of keywords during a verbal recitation of the selected story, audio content with a sound effect that animates an event described by the at least one keyword;

receiving user input to initiate verbal recitation of the selected story;

detecting the at least one keyword during the verbal recitation of the selected story, wherein each of the at least one keyword is detected based on the lexicon associated with the selected story, respective one or more words from the selected story that are being verbally recited, and a location of the respective one or more words within the selected story; and

performing the at least one action responsive to detecting the at least one keyword during the verbal recitation of the selected story, wherein the at least one action comprises playing the audio content with the sound effects that animates the event described by the at least one keyword in the selected story.

16. The non-transitory computer-readable medium of claim 15 , wherein the client device is a voice activated home assistant device.

17. The non-transitory computer-readable medium of claim 15 , wherein the lexicon further comprises a total number of words in the selected story, wherein data indicating the location of each keyword within the selected story is represented by a word count associated with a respective keyword in view of the total number of words, and wherein detecting the at least one keyword further comprises:

sending data defining the verbal recitation of the selected story to the server, the sent data comprising locations of respective verbally recited words within the selected story, wherein each location is represented by a corresponding word count; and

receiving from the server correspondence between the data defining the verbal recitation of the selected story and the at least one keyword.

18. The non-transitory computer-readable medium of claim 15 , wherein the detecting comprises using a matching algorithm to match an audio profile of the at least one keyword with an audio profile of the user input.

19. The non-transitory computer-readable medium of claim 15 , wherein the verbal recitation comprises playback of a recorded narration of the story, the recorded narration received from the server in response to the user selection of the story.

20. The non-transitory computer-readable medium of claim 15 , wherein the verbal recitation is a narration of the selected story by a user of the client device, and wherein the narration of the selected story is received via a microphone of the client device.

Assignments (2)
CHANGE OF NAME Recorded Mar 22, 2019
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 048680/0966 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2019
From: ROTHERA, ALEXANDER; LANGE, SCOTT D.; DROP, JENNIFER
To: GOOGLE INC.
Reel/Frame 048350/0236 →
Continuity (2)
Continuation 15289806 · Oct 10, 2016
Related Publication 20190158632A1 · May 23, 2019
Cited By (1)
US 12,278,859