IP Library › Granted Patent US 9,521,239
Granted Patent B2
US 9,521,239 · App. 14/260,922 · Granted Dec 13, 2016

Apparatus and method for automatic discovery and suggesting personalized gesture control based on user's habit and context

Inventor: Zhiyun Li (Kenmore, WA)
Assignee: Samsung Electronics Co., Ltd.
H04M1/72533H04M11/007H04W4/025
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,521,239
App. No.
14/260,922
Granted
Dec 13, 2016
Kind
B2
Abstract

An apparatus and method for automatic discovery and suggesting personalized gestures from mobile terminal to control a plurality of devices in smart home system, based on user's habit and context. The method includes recording, via a mobile terminal, a context of the mobile terminal, recoding, via home system, actions taken by the plurality of devices in the home system, recognizing as a gesture command from a correlation of at least a portion of the recorded context of the user's mobile terminal with the recorded actions taken by the plurality of devices in the home system, and controlling at least one of the plurality of devices based a repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system.

Claims (35)

1. A method for integrated discovery and control of a plurality of devices based on movement, context and/or gesture, the method comprising:

recording, via a mobile terminal, a context of a user of the mobile terminal;

recording, via home system, actions taken by the plurality of devices in the home system;

recognizing as a gesture command from a correlation of at least a portion of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system; and

controlling at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system,

wherein the recorded context of the user of the mobile terminal includes macro movements recorded as a map movement and micro movements recorded as other movements of the user of the mobile terminal in relation to the user's external environment not recorded as the map movement.

2. The method of claim 1 , further comprising:

suggesting the gesture command based on at least a portion of the recognition of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system; and

confirming, in response to the suggesting of the gesture command, the gesture command, the gesture command enables the controlling of the at least one of the plurality of devices in the home system.

3. The method of claim 1 , wherein the recording of the context of the user of the mobile terminal comprises recording at least a time, usage history and a physical movement of the mobile terminal.

4. The method of claim 3 , wherein the physical movement of the mobile terminal is determined via at least one of a Global Position System (GPS), WIFI hotspot, cell tower triangulation and a motion sensor.

5. The method of claim 2 , wherein the correlation of the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system comprises:

identifying the recorded actions taken by the at least one of the plurality of devices in the home system;

recognizing via a pattern classifier, a repeated pattern from the recorded context of the mobile terminal in correlation with a repeated pattern from the recorded actions taken by the at least one of the plurality of devices in the home system; and

matching, based on the pattern classifier, an action taken by the at least one of the plurality of devices in the home system with the recognized repeated pattern from the history recorded by the mobile terminal.

6. The method of claim 1 , wherein the controlling of the at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system comprises:

causing, when it is determined that a current context of a user of the mobile terminal are the same as the recorded context, and that the gesture command associated with the recorded patterns of the user of the mobile terminal, execution of an action associated with the gesture command of the user of the mobile terminal.

7. The method of claim 6 , wherein the repeated context of the user of the mobile terminal is walking up a walk way to a home of the user, stepping up several steps and pausing at a top of the several steps, and

wherein the controlling of at least one of the plurality of devices based on the user's repeated context comprises automatically opening a door of the user's home by a home system in communication with the mobile device.

8. An apparatus for integrated discovery and control of a plurality of devices based on movement and/or gesture, the apparatus comprising:

a mobile device recording unit configured to record a context of a user of the mobile terminal;

a home system recoding unit configured to record actions taken by the plurality of devices in the home system; and

a controller configured to:

recognize as a gesture command from a correlation of at least a portion of the recorded context of the user of the mobile terminal with the recorded actions taken by the at least one of the plurality of devices in the home system, and

control at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system,

wherein the recorded context of the user of the mobile terminal includes macro movements recorded as a map movement and micro movements recorded as other movements of the user of the mobile terminal in relation to the user's external environment not recorded as the map movement.

9. The apparatus of claim 8 , further comprising:

a gesture unit configured to suggest the gesture command based on at least a portion of the recognition of the recorded context of the user of the mobile terminal with the recorded actions taken by the at least one of plurality of devices in the home system; and

a confirmation unit configured to confirm, in response to the suggesting of the gesture command, the gesture command, the gesture command enables the controlling of the at least one the plurality of devices in the home system.

10. The apparatus of claim 8 , wherein mobile device recording unit records at least a time, usage history and a physical movement of the mobile terminal.

11. The apparatus of claim 10 , wherein the physical movement of the mobile terminal is determined via at least one of a Global Position System (GPS), WIFI hotspot, cell tower triangulation and a motion sensor.

12. The apparatus of claim 9 , wherein the controller correlating the recorded context of the user of the mobile terminal with the recorded actions taken by the plurality of devices in the home system comprises identifying the recorded actions taken by the at least one of the plurality of devices in the home system, recognizing via a pattern classifier, a repeated pattern from the recorded context of the mobile terminal in correlation with a repeated pattern from the recorded actions taken by the at least one of the plurality of devices in the home system, and matching, based on the pattern classifier, an action taken by the at least one of the plurality of devices in the home system with the recognized repeated pattern from the history recorded by the mobile terminal.

13. The apparatus of claim 8 , wherein the controller controlling the at least one of the plurality of devices based on repetition of the gesture command, the controlling repeating the actions taken by the at least one of the plurality of devices in the home system comprises causing, when it is determined that a current context of a user of the mobile terminal are the same as the recorded context, and that the gesture command associated with the recorded patterns of the user of the mobile terminal, execution of an action associated with the gesture command of the user of the mobile terminal.

14. The apparatus of claim 13 , wherein, the controller upon detecting the repeated context of the user of the mobile terminal as walking up a walk way to a home of the user, stepping up several steps and pausing at a top of the several steps, controls at least one of the plurality of devices based on the user's confirming of the action is an automatic opening of a door of the user's home by a home system in communication with the mobile device.

15. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to perform the method of claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2014
From: LI, ZHIYUN
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 032750/0800 →
Continuity (1)
Related Publication 20150312398A1 · Oct 29, 2015