IP Library Granted Patent US 10,380,852
Granted Patent B2
US 10,380,852 · App. 15/895,910 · Granted Aug 13, 2019

Systems, methods, and devices for activity monitoring via a home assistant

Inventor: Bryan Horling (Sunnyvale, CA)
Assignee: GOOGLE LLC
G08B13/1672G05B15/02G08B15/002G08B25/008H05B37/0227G05B2219/2642
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,380,852
App. No.
15/895,910
Granted
Aug 13, 2019
Kind
B2
Abstract

The various implementations described herein include methods, devices, and systems for monitoring activity in a home environment. In one aspect, a method performed at a voice-assistant device includes: (1) while operating in an assistant mode, receiving an occupant voice command to operate in a monitoring mode; (2) in response to the command, transitioning to operating in the monitoring mode; (3) obtaining one or more monitoring criteria; (4) while operating in the monitoring mode, detecting a sound; (5) obtaining a determination as to whether the sound meets the one or more monitoring criteria; and (6) in accordance with a determination that the sound meets the one or more monitoring criteria: (a) obtaining a classification of the sound; and (b) based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

Claims (75)

1. A method for monitoring activity in a home environment, comprising:

at a computer system having a voice-activated user interface, one or more microphones, one or more speakers, one or more processors, and memory, wherein the computer system has at least two distinct operating modes, including an assistant mode and a monitoring mode:

while operating in the assistant mode, receiving an occupant voice command to operate in the monitoring mode;

in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;

obtaining one or more monitoring criteria;

while operating in the monitoring mode, detecting a sound;

obtaining a determination as to whether the sound meets the one or more monitoring criteria; and

in accordance with a determination that the sound meets the one or more monitoring criteria:

obtaining a classification of the sound; and

based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

2. The method of claim 1 , wherein the plurality of simulated occupant responses includes one or more of:

a simulated animal response;

a pre-recorded occupant response;

a request for identification; and

a simulated human response.

3. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, performing one or more of:

adjusting a lighting level of the home environment;

adjusting a user interface of the computer system to indicate that the sound meeting the one or more predefined criteria was detected;

storing the sound;

enabling a home device; and

sending an alert.

4. The method of claim 1 , further comprising, in accordance with the determination that the sound does not meet the one or more monitoring criteria, forgoing obtaining the classification of the sound.

5. The method of claim 1 , further comprising, based on the sound having a second sound classification, forgoing emitting a simulated occupant response via the one or more speakers.

6. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, enabling a remote occupant of the home environment to perform one or more of: listen to the sound, listen to the home environment via the one or more microphones.

7. The method of claim 1 , wherein obtaining the determination as to whether the sound meets the one or more predefined criteria includes obtaining a determination as to whether the sound is an expected sound of the home environment.

8. The method of claim 7 , further comprising obtaining a plurality of previously-recorded home environment sounds; and

wherein obtaining the determination that the sound is not an expected sound of the home environment comprises comparing the sound to the plurality of previously-recorded home environment sounds.

9. The method of claim 1 , wherein obtaining the classification of the sound comprises obtaining a determination as to whether the sound comprises a voice of an occupant of the home environment.

10. The method of claim 1 , wherein obtaining the classification of the sound includes obtaining a determination as to whether an occupant of the home environment is expected to currently be within the home environment.

11. The method of claim 10 , further comprising, in accordance with a determination that the occupant of the home environment is expected to currently be within the home environment, emitting a second simulated occupant response via the one or more speakers, the second simulated occupant response distinct from the simulated occupant response.

12. The method of claim 1 , further comprising:

receiving a second occupant command to operate the computer system in a learning mode;

in response to the second occupant command, transitioning to operating in the learning mode;

while operating in the learning mode, analyzing sounds of the home environment to identify a plurality of expected sounds within the home environment.

13. The method of claim 1 , further comprising:

while operating in the monitoring mode, receiving a second occupant command to operate in the assistant mode; and

in accordance with the second occupant command, transitioning to operating in the assistant mode.

14. The method of claim 13 , further comprising:

verifying that the second occupant command comprises a command from an authorized user; and

transitioning to operating in the assistant mode in accordance with verification of the second occupant command.

15. The method of claim 1 , wherein the computer system is coupled to one or more additional computer systems within the home environment; and

the method further comprises, in accordance with the determination that the sound meets the one or more monitoring criteria, requesting that at least one of the one or more additional computer systems simulates an occupant response to the sound.

16. The method of claim 1 , wherein the computer system is coupled to one or more additional computer system within the home environment; and

the method further comprises:

receiving notification of a second sound meeting one or more predefined criteria from at least one of the one or more additional computer systems; and

in response to receiving notification of the second sound, emitting a second simulated occupant response via the one or more speakers.

17. The method of claim 1 , further comprising, while in the monitoring mode, mimicking occupant activity within the home environment.

18. The method of claim 1 , further comprising:

while in the monitoring mode, anticipating a particular sound within the home environment at a particular time; and

in accordance with a determination that the particular sound did not occur at the particular time, emitting a second simulated occupant response via the one or more speakers.

19. A computer system comprising:

a voice-activated user interface;

one or more microphones;

one or more speakers;

one or more processors coupled to the one or more microphones and the one or more speakers; and

memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

operating in at least two distinct modes, including an assistant mode and a monitoring mode;

while operating in the assistant mode, receiving an occupant voice command to operate in the monitoring mode;

in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;

obtaining one or more monitoring criteria;

while operating in the monitoring mode, detecting a sound;

obtaining a determination as to whether the sound meets the one or more monitoring criteria; and

in accordance with a determination that the sound meets the one or more monitoring criteria:

obtaining a classification of the sound; and

based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system having one or more microphones and one or more speakers, cause the system to:

operate in at least two distinct modes, including an assistant mode and a monitoring mode;

while operating in the assistant mode, receive an occupant voice command to operate in the monitoring mode;

in response to the occupant command, transition from operating in the assistant mode to operating in the monitoring mode;

obtain one or more monitoring criteria;

while operating in the monitoring mode, detect a sound;

obtain a determination as to whether the sound meets the one or more monitoring criteria; and

in accordance with a determination that the sound meets the one or more monitoring criteria:

obtain a classification of the sound; and

based on sound having a first sound classification, emit a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2018
From: HORLING, BRYAN
To: GOOGLE LLC
Reel/Frame 044928/0247 →
Continuity (2)
Provisional Application 62505761 · May 12, 2017
Related Publication 20180330589A1 · Nov 15, 2018
Cited By (7)
US 12,322,397 US 12,334,078 US 12,361,930 US 12,374,202 US 12,424,221 US 12,562,047 US 12,567,408