IP Library › Granted Patent US 10,270,609
Granted Patent B2
US 10,270,609 · App. 14/725,989 · Granted Apr 23, 2019

Automatically learning and controlling connected devices

Inventors: Ashutosh Saxena (Cupertino, CA); Hema Swetha Koppula (Mountain View, CA); Chenxia Wu (Menlo Park, CA); Ozan Sener (Palo Alto, CA)
Assignee: BrainofT Inc.
H04L12/2803G05B15/02G06N99/005H04L12/2829H04L67/12H04L67/18G05B2219/2642
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,270,609
App. No.
14/725,989
Granted
Apr 23, 2019
Kind
B2
Abstract

A first input is received from a plurality of sensors. A first state including a first location based on the first input is determined. The first state is associated with a first probability. A second input is received from the plurality of sensors. A second state including a second location is determined based on the second input associated with a second probability. It is determined that the second state corresponds to an actual state based on a transition model and the second probability. The transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state. A rule to change a state of at least one network connected device is triggered based on the second state.

Claims (42)

1. A system for controlling network connected devices, including:

a processor configured to:

receive a first input from a plurality of sensors;

determine a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receive a second input from the plurality of sensors;

determine a second state including a second location based on the second input associated with a second probability;

determine that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

trigger a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions.

2. The system of claim 1 , wherein the plurality of sensors includes a camera.

3. The system of claim 1 , wherein the first state is a first state of a human subject and the second state is a second state of the human subject.

4. The system of claim 1 , wherein the first state includes a first event and the second state includes a second event.

5. The system of claim 4 , wherein the second activity is one of a plurality of human activities predefined to be recognizable.

6. The system of claim 4 , wherein the second activity has been detected by computer vision analysis of a video image.

7. The system of claim 1 , wherein the second probability identifies a likelihood the second state is the actual state.

8. The system of claim 1 , wherein the first state has been identified as a previous valid state.

9. The system of claim 1 , wherein determining that the second state corresponds to the actual state includes determining that the second probability meets a threshold value.

10. The system of claim 1 , wherein the rule is based on a previous observation associated with the second state.

11. The system of claim 1 , wherein the rule is based on a feedback from a previous trigger of the rule.

12. The system of claim 1 , wherein the second state triggers a plurality of rules that conflict and the triggered rule has been selected for triggering based on a comparison of priority values of the plurality of rules.

13. The system of claim 1 , wherein the transition model was generated using deep learning to analyze a history of state transitions.

14. The system of claim 1 , wherein the processor is further configured to determine that a first sensor data and a second sensor data are associated with a same detected subject.

15. The system of claim 1 , wherein determining that the second state corresponds to the actual state includes determining a correlation between components of the second state.

16. The system of claim 1 , wherein the first state and the second state have been determined using machine learning.

17. The system of claim 1 , wherein the rule was automatically generated based on observations detected using the plurality of the sensors.

18. The system of claim 1 , wherein the first location identifies a physical location within a building.

19. The system of claim 1 , wherein the first input and the second input are included in streaming inputs received from the plurality of sensors.

20. The system of claim 1 , wherein changing the state of the at least one network connected device based on the second state includes sending a command to the at least one network connected device.

21. A method for controlling network connected devices, including:

receiving a first input from a plurality of sensors;

determining a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receiving a second input from the plurality of sensors;

determining a second state including a second location based on the second input associated with a second probability;

using a processor to determine that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

triggering a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state.

22. A computer program product for controlling network connected devices, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a first input from a plurality of sensors;

determining a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receiving a second input from the plurality of sensors;

determining a second state including a second location based on the second input associated with a second probability;

determining that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

triggering a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE EXECUTION DATE FOR THE SECOND LISTED ASSIGNOR PREVIOUSLY RECORDED ON REEL 036149 FRAME 0901. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT Recorded Apr 15, 2016
From: SAXENA, ASHUTOSH; KOPPULA, HEMA SWETHA; WU, CHENXIA; SENER, OZAN
To: BRAINOFT INC.
Reel/Frame 038440/0984 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2015
From: SAXENA, ASHUTOSH; KOPPULA, HEMA SWETHA; WU, CHENXIA; SENER, OZAN
To: BRAINOFT INC.
Reel/Frame 036149/0901 →
Continuity (2)
Provisional Application 62120240 · Feb 24, 2015
Related Publication 20160248847A1 · Aug 25, 2016