IP Library Granted Patent US 10,664,689
Granted Patent B2
US 10,664,689 · App. 15/587,872 · Granted May 26, 2020

Determining user activity based on eye motion

Inventor: Daye Yang (Beijing, CN)
Assignee: LENOVO (BEIJING) CO., LTD.
G06K9/00335G06F3/013G06K9/0061G06K9/00604G06K9/4676G06Q10/00G06Q30/0201G06T7/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,664,689
App. No.
15/587,872
Granted
May 26, 2020
Kind
B2
Abstract

An information processing method is provided. The information processing method includes acquiring a motion state of eyes of a user to form eye motion data and record a first acquisition time of the eye motion data; extracting position data of the user's eyeballs from the eye motion data; and capturing user behavior activity data to record a second acquisition time of the user behavior activity data. The method also includes, based on the first acquisition time and the second acquisition time, determining a correspondence relationship between the position data of the user's eyeballs and the user behavior activity data; and, based on the correspondence relationship and a current eye motion, determining a current user behavior activity.

Claims (58)

1. A method, comprising:

processing, by a processor, images of eyes of a user to obtain eye motion data about motion stares of the eyes of the user;

extracting, by the processor, position data of user's eyeballs from the eye motion data;

encoding, by the processor, the position data of the user's eyeballs to obtain coding sequences each following a preset coding rule and including a sequence of codes representing various information regarding positions of the user's eyeballs;

processing, by the processor, images of a body of the user to obtain user behavior activity data, the user behavior activity data indicating behavior activities of the user and being based on motion states of the body of the user other than the eyes of the user;

correlating, by the processor, the coding sequences to the user behavior activity data to obtain a correspondence relationship;

combining two of the coding sequences that have an association of the motion states of the eyes with the correspondence relationship;

recording a correspondence probability and an interval sequence number of the correspondence relationship, wherein the correspondence probability is a probability that the motion states of the eyes corresponding to the two of the coding sequences occur together, and the interval sequence number of the correspondence relationship is an average interval number of coding sequences between the two of the coding sequences or a number of interval coding sequences with a highest frequency when the two of the coding sequences occur together; and

based on the correspondence relationship and a current eye motion, determining, by the processor, a current user behavior activity.

2. The method according to claim 1 , wherein:

each of the coding sequences includes an action identifier indicating an action of the eyes of the user, a view region identifier indicating a region from a divided vision field of the user, and a state identifier indicating a time length.

3. The method according to claim 2 , wherein a vision field within the user's eyes is divided into a plurality of regions and the view region identifier indicates a region at which the user's eyes are located or to which the user's eyes move.

4. The method according to claim 1 , wherein correlating the coding sequences to the user behavior activity data further includes:

dividing the coding sequences into a plurality of data sets each including one or more of the coding sequences corresponding to the position data acquired in a time period;

arranging the one or more of the coding sequences in one of the data sets according to a preset rule to obtain a preset coding term corresponding to the one of the data sets; and

establishing a correspondence relationship between the preset coding term and the user behavior activity data.

5. The method according to claim 4 , further including:

analyzing the user behavior activity data to determine different user behavior activities;

wherein the coding sequences are divided according to the different user behavior activities.

6. The method according to claim 5 , wherein:

different ones of the data sets correspond to different time periods; and

the coding sequences in each of the data sets are sequentially sorted in a chronological order.

7. The method according to claim 5 , wherein the preset coding term corresponding to the one of the data sets is determined based on occurrence frequencies of the coding sequences in the one of the data sets and a correspondence between any two of the coding sequences in the one of the data sets.

8. An electronic device, comprising a processor, and a memory including computer instructions executable by the processor to:

process images of eyes of a user to obtain eye motion data about motion states of the eyes of the user;

extract position data of user's eyeballs from the eye motion data;

encode the positional data of the user's eyeballs to obtain coding sequences, each following a preset coding rule and including a sequence of codes representing various information regarding positions of the user's eyeballs;

process images of a body of the user to obtain user behavior activity data, the user behavior activity data indicating behavior activities of the user and being based on motion states of the body of the user other than the eyes of the user;

correlate the coding sequences to the user behavior activity data to obtain a correspondence relationship;

combine two of the coding sequences that have an association of the motion states of the eyes with a correspondence relationship;

record a correspondence probability and an interval sequence number of the correspondence relationship, wherein the correspondence probability is a probability that the motion states of the eyes corresponding to the two of the coding sequences occur together, and the interval sequence number of the correspondence relationship is an average interval number of coding sequences between the two of the coding sequences or a number of interval coding sequences with a highest frequency when the two of the coding sequences occur together; and

determine a current user behavior activity based on the correspondence relationship and a current eye motion.

9. The electronic device according to claim 8 , wherein:

each of the coding sequences includes an action identifier indicating an action of the eyes of the user, a view region identifier indicating a region from a divided vision field of the user, and a state identifier indicating a time length.

10. The electronic device according to claim 8 , wherein the computer instructions are further executed by the processor to:

divide the coding sequences into a plurality of data sets each including one or more of the coding sequences corresponding to the position data acquired in a time period;

arrange the one or more of the coding sequences in one of the data sets according to a preset rule to obtain a preset coding term corresponding to the one of the data sets; and

establish a correspondence relationship between the preset coding term and the user behavior activity data.

11. The electronic device according to claim 10 , wherein:

the computer instructions are further executable by the processor to analyze the user behavior activity data to determine different user behavior activities; and

the coding sequences are divided according to the different user behavior activities.

12. The electronic device according to claim 11 , wherein:

different ones of the data sets correspond to different time periods; and

the coding sequences in each of the data sets are sequentially sorted in a chronological order.

13. The electronic device according to claim 11 , wherein the preset coding term corresponding to the one of the data sets is determined based on occurrence frequencies of the coding sequences in the one of the data sets and a correspondence between any two of the coding sequences in the one of the data sets.

14. A method, comprising:

processing, by a processor, images of eyes of a user to obtain eye motion data about motion states of the eyes of the user;

extracting, by the processor, position data of user's eyeballs from the eye motion data;

encoding, by the processor, the position data of the user's eyeballs to obtain coding sequences each following a preset coding rule and including a sequence of codes representing various information regarding positions of the user's eyeballs;

processing, by the processor, images of a body of the user to obtain user behavior activity data, the user behavior activity data indicating behavior activities of the user and being based on motion states of the body of the user other than the eyes of the user;

analyzing the user behavior activity data to determine different user behavior activities, wherein the coding sequences are divided according to the different user behavior activities;

correlating, by the processor, the coding sequences to the user behavior activity data to obtain a correspondence relationship, including:

dividing the coding sequences into a plurality of data sets each including one or more of the coding sequences corresponding to the position data acquired in a time period;

arranging the one or more of the coding sequences in one of the data sets according to a preset rule to obtain a preset coding term corresponding to the one of the data sets based on occurrence frequencies of the coding sequences in the one of the data sets and a correspondence between any two of the coding sequences in the one of the data sets; and

establishing a correspondence relationship between the preset coding term and the user behavior activity data;

combining two of the coding sequences that have an association of the motion states of the eyes with the correspondence relationship;

recording a correspondence probability and an interval sequence number of the correspondence relationship, wherein the correspondence probability is a probability that the motion states of the eyes corresponding to the two of the coding sequences occur together, and the interval sequence number of the correspondence relationship is an average interval number of coding sequences between the two of the coding sequences or a number of interval coding sequences with a highest frequency when the two of the coding sequences occur together; and

based on the correspondence relationship and a current eye motion, determining, by the processor, a current user behavior activity.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2017
From: YANG, DAYE
To: LENOVO (BEIJING) CO., LTD.
Reel/Frame 042255/0301 →
Priority Claims (1)
CN 2016 1 0482953 · Jun 27, 2016 · national
Continuity (1)
Related Publication 20170372131A1 · Dec 28, 2017