IP Library › Granted Patent US 10,152,123
Granted Patent B2
US 10,152,123 · App. 15/519,934 · Granted Dec 11, 2018

Method and system for detecting objects of interest

Inventors: Tore Meyer (Munich, DE); Stephan Odörfer (Munich, DE)
Assignee: 4TIITOO GMBH
G06F3/013G06F3/0482G06F17/30247G06F17/30256G06F17/30268G06F17/30274
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,152,123
App. No.
15/519,934
Granted
Dec 11, 2018
Kind
B2
Abstract

A method for detecting objects ( 28 ) of interest comprises: receiving gaze tracking data ( 30 ) of a person looking at a collection of objects ( 28 ), the gaze tracking data ( 30 ) comprising information, at which points the person is looking; determining a map ( 42 ) of objects in the field of view of the person, the map ( 42 ) of objects indicating, at which position ( 44 ) an object in the collection of objects is arranged; determining gaze interaction events ( 48 ) for objects in the collection of objects by matching the gaze tracking data ( 30 ) to the map ( 42 ) of objects, wherein a gaze interaction event ( 48 ) for an object indicates that a specific gazing interaction has occurred with respect to the object; determining category characteristics ( 56 ) for at least those objects ( 28 ) having gaze interaction events ( 48 ), wherein a category characteristic ( 56 ) describes the object with respect to a specific object category ( 58 ); and determining at least one category characteristic ( 56 ) of interest in at least one object category ( 58 ) by finding at least one category characteristic ( 56 ) which is assigned to a plurality of objects ( 28 ), which have gaze interaction events ( 48 ) indicating the person is interested in these objects ( 28 ).

Claims (37)

1. A method for detecting objects of interest, the method comprising:

receiving gaze tracking data of a person looking at a collection of objects, the gaze tracking data comprising information, at which points the person is looking;

determining a map of objects in the field of view of the person, the map of objects indicating, at which position an object in the collection of objects is arranged;

determining gaze interaction events for objects in the collection of objects by matching the gaze tracking data to the map of objects, wherein a gaze interaction event for an object indicates that a specific gazing interaction has occurred with respect to the object and wherein a gaze interaction event is based on a model of a peripheral view of the person;

determining category characteristics for objects having gaze interaction events, wherein a category characteristic describes the object with respect to a specific object category;

determining at least one category characteristic of interest in at least one object category by finding at least one category characteristic which is assigned to a plurality of objects, which have gaze interaction events indicating the person is interested in these objects

determining a negative involvement level for an object from the gaze interaction events of the object based on the model of the peripheral view of the person, wherein objects with a negative involvement level higher than a negative involvement level threshold are excluded for a selection of objects.

2. The method of claim 1 , wherein category characteristics of objects are clustered and the at least one category characteristic of interest is determined by selecting at least one cluster of category characteristics representing the at least one category characteristic of interest.

3. The method of claim 1 , wherein at least two different types of gaze interaction events are determined for an object.

4. The method of claim 1 wherein the peripheral view is modeled with an area surrounding a focus point and excluding the focus point.

5. The method of claim 1 , wherein category characteristics for at least two different types of object categories are determined for an object.

6. The method of claim 1 , wherein:

the category characteristics for the collection of objects are precalculated and stored together with the objects; and/or

wherein the category characteristics are determined by evaluating object data of the objects.

7. The method of claim 1 , wherein object categories are determined by evaluating object data of objects from the group of interesting objects.

8. The method of claim 1 , further comprising:

determining an involvement level for an object from the gaze interaction events of the object, wherein only objects with an involvement level higher than an involvement level threshold are considered for the selection of objects.

9. The method of claim 1 , wherein:

the collection of objects is displayed on a display device in the field of view of the person; and/or

after the at least one category characteristic of interest has been determined, additional objects having the at least one characteristic of interest are displayed on the display device.

10. The method of claim 1 , wherein:

the map of objects is generated from outlines of objects; and/or

the map of objects is generated from image data received from a camera recording the field of view of the person.

11. The method of claim 1 , wherein:

the collection of objects comprises at least one of: images, words, text blocks, icons, formulas, pages, sections of a document, 3D objects, shapes, patterns; and/or

the object categories are content descriptors; and/or

the category characteristics are values of content descriptors.

12. A non-transitory computer-readable medium, in which a computer program for detecting objects of interest is stored, which, when being executed by a processor, is adapted to carry out the steps of: receiving gaze tracking data of a person looking at a collection of objects, the gaze tracking data comprising information, at which points the person is looking; determining a map of objects in the field of view of the person, the map of objects indicating, at which position an object in the collection of objects is arranged; determining gaze interaction events for objects in the collection of objects by matching the gaze tracking data to the map of objects, wherein a gaze interaction event for an object indicates that a specific gazing interaction has occurred with respect to the object and wherein a gaze interaction event is based on a model of a peripheral view of the person; determining category characteristics for objects having gaze interaction events, wherein a category characteristic describes the object with respect to a specific object category; determining at least one category characteristic of interest in at least one object category by finding at least one category characteristic which is assigned to a plurality of objects, which have gaze interaction events indicating the person is interested in these objects; determining a negative involvement level for an object from the gaze interaction events of the object based on the model of the peripheral view of the person, wherein objects with a negative involvement level higher than a negative involvement level threshold are excluded for a selection of objects.

13. A gaze tracking system, comprising:

a gaze tracking device for generating gaze tracking data; and

an object detection device adapted for performing the steps of:

receiving gaze tracking data of a person looking at a collection of objects, the gaze tracking data comprising information, at which points the person is looking;

determining a map of objects in the field of view of the person, the map of objects indicating, at which position an object in the collection of objects is arranged;

determining gaze interaction events for objects in the collection of objects by matching the gaze tracking data to the map of objects, wherein a gaze interaction event for an object indicates that a specific gazing interaction has occurred with respect to the object and wherein a gaze interaction event is based on a model of a peripheral view of the person;

determining category characteristics for objects having gaze interaction events, wherein a category characteristic describes the object with respect to a specific object category;

determining at least one category characteristic of interest in at least one object category by finding at least one category characteristic which is assigned to a plurality of objects, which have gaze interaction events indicating the person is interested in these objects;

determining a negative involvement level for an object from the gaze interaction events of the object based on the model of the peripheral view of the person, wherein objects with a negative involvement level higher than a negative involvement level threshold are excluded for a selection of objects.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2017
From: MEYER, TORE; ODORFER, STEPHAN
To: 4TIITOO GMBH
Reel/Frame 043234/0337 →
Priority Claims (1)
EP 14191120 · Oct 30, 2014 · regional
Continuity (1)
Related Publication 20170336865A1 · Nov 23, 2017