IP Library Granted Patent US 10,628,714
Granted Patent B2
US 10,628,714 · App. 15/682,407 · Granted Apr 21, 2020

Entity-tracking computing system

Inventors: Vivek Pradeep (Redmond, WA); Pablo Luis Sala (Bothell, WA); John Guido Atkins Weiss (Lake Forest Park, WA); Moshe Randall Lutz (Bellevue, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06K9/726A61B5/0205A61B5/0507A61B5/117A61B5/1113A61B5/7475G01S5/18G01S5/28G01S13/726G06F1/324G06F1/3206G06F1/3231G06F3/011G06F3/017G06F3/0304G06F3/0482G06F3/04842G06F3/167G06F17/271G06F17/279G06F21/32G06F21/35G06K9/00214G06K9/00255G06K9/00261G06K9/00288G06K9/00295G06K9/00342G06K9/00362G06K9/00711G06K9/00771G06K9/00973G06K9/6254G06K9/6255G06K9/6289G06K9/6296G06N5/025G06N5/047G06N20/00G06T7/248G06T7/292G06T7/60G06T7/70G06T7/74G07C9/00111G08B13/1427G10L15/02G10L15/063G10L15/08G10L15/18G10L15/1815G10L15/1822G10L15/19G10L15/22G10L15/24G10L15/26G10L15/28G10L15/32G10L17/04G10L17/08G10L25/51H04L51/02H04L63/102H04L67/12H04L67/22H04N5/23219H04N5/332H04N7/181H04N7/188H04N21/231H04N21/42203H04N21/44218H04N21/44222H04R1/406H04R3/005H04W4/029H04W4/33A61B5/05A61B5/1118G01S5/16G01S11/14G01S13/38G01S13/867G01S13/888G06F3/0488G06F16/70G06F2203/0381G06F2221/2111G06K2209/09G06N3/0445G06T2207/10016G06T2207/10024G06T2207/20101G06T2207/30196G06T2207/30201G06T2207/30204G06T2207/30232G07C9/00134G08B29/186G10L17/00G10L2015/0635G10L2015/088G10L2015/223G10L2015/225G10L2015/228H04N5/247Y02D10/126Y02D10/173
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,628,714
App. No.
15/682,407
Filed
Aug 21, 2017
Granted
Apr 21, 2020
Kind
B2
Art Unit
3665
USPC
701/495
Abstract

An entity-tracking computing system receives sensor information from a plurality of different sensors. The positions of entities detected by the various sensors are resolved to an environment-relative coordinate system so that entities identified by one sensor can be tracked across the fields of detection of other sensors.

Claims (48)

1. An entity-tracking computing system, comprising:

a logic machine;

a communication subsystem; and

a storage machine holding instructions executable by the logic machine to:

via the communication subsystem, communicatively couple the entity-tracking computing system to a plurality of sensors each having a field-of-detection (FOD) within an environment;

maintain an environment-relative coordinate system to which the FOD of each sensor of the plurality of sensors is mapped, where mapping the FOD of a particular sensor of the plurality of sensors to the environment-relative coordinate system includes resolving a position of a calibration device relative to the particular sensor, and identifying a correspondence between the position of the calibration device and the FOD of the particular sensor;

receive, via the communication subsystem and from a first sensor of the plurality of sensors, an indication of presence of a first entity within a FOD of the first sensor at a sensor-relative position within a sensor-relative coordinate system of the first sensor;

translate the sensor-relative position of the first entity to an environment-relative position of the first entity within the environment-relative coordinate system;

assign a unique entity identifier to the first entity;

receive, from a second sensor of the plurality of sensors, an indication of presence of a second entity within a FOD of the second sensor at a sensor-relative position within a sensor-relative coordinate system of the second sensor;

translate the sensor-relative position of the second entity to an environment-relative position of the second entity within the environment-relative coordinate system; and

based on determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity, assign the unique entity identifier to the second entity.

2. The entity-tracking computing system of claim 1 , where resolving the position of the calibration device includes receiving from the calibration device information useable to assess the position of the calibration device within the environment-relative coordinate system.

3. The entity-tracking computing system of claim 2 , where the particular sensor incorporates a positioning marker, and the instructions are further executable to receive, from the calibration device, information useable to assess a position of the positioning marker within the environment-relative coordinate system.

4. The entity-tracking computing system of claim 2 , where the calibration device incorporates a positioning marker, and the instructions are further executable to receive, from the particular sensor, information useable to assess a position of the positioning marker within a sensor-relative coordinate system of the particular sensor.

5. The entity-tracking computing system of claim 1 , where the first entity is unidentified, and where the unique entity identifier is a generic identifier.

6. The entity-tracking computing system of claim 5 , where data received from the second sensor is useable to identify the second entity as a previously-identified human, and the instructions are further executable to replace the generic identifier with a previously-assigned entity identifier associated with the previously-identified human.

7. The entity-tracking computing system of claim 1 , where the first entity is previously-identified human, and where the unique entity identifier is a previously-assigned entity identifier associated with the previously-identified human.

8. The entity-tracking computing system of claim 1 , where the first sensor is a camera, and where the data received from the camera includes information usable to identify a human face in the first FOD.

9. The entity-tracking computing system of claim 1 , where data received from the second sensor is not sufficient to identify the second entity without the data received from the first sensor.

10. The entity-tracking computing system of claim 1 , where determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity includes determining that the environment-relative position of the second entity is within a threshold distance of the environment-relative position of the first entity according to the environment-relative coordinate system.

11. The entity-tracking computing system of claim 1 , where the instructions are further executable to predict a path of the first entity after the first entity leaves the FOD of the first sensor, and determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity includes determining that the environment-relative position of the second entity is consistent with the path after the second entity enters the FOD of the second sensor.

12. The entity-tracking computing system of claim 1 , where determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity includes determining that one or more identifying characteristics of the second entity are within a threshold similarity of one or more identifying characteristics of the first entity.

13. The entity-tracking computing system of claim 1 , where the plurality of sensors includes a plurality of cameras.

14. The entity-tracking computing system of claim 13 , where the plurality of cameras includes one or more infrared (IR) cameras.

15. The entity-tracking computing system of claim 13 , where the plurality of cameras includes one or more depth cameras.

16. The entity-tracking computing system of claim 13 , where the plurality of cameras includes one or more downward-facing overhead cameras.

17. The entity-tracking computing system of claim 1 , where the plurality of sensors includes a radar sensor.

18. A method for tracking entities in an environment, comprising:

maintaining an environment-relative coordinate system to which a field-of-detection (FOD) of each of a plurality of sensors is mapped, each of the plurality of sensors having a FOD within the environment, where mapping the FOD of a particular sensor of the plurality of sensors to the environment-relative coordinate system includes resolving a position of a calibration device relative to the particular sensor, and identifying a correspondence between the position of the calibration device and the FOD of the particular sensor;

receiving an indication of presence of a first entity within a FOD of a first sensor of the plurality of sensors at a sensor-relative position within a sensor-relative coordinate system of the first sensor;

translating the sensor-relative position of the first entity to an environment-relative position of the first entity within the environment-relative coordinate system;

assigning a unique entity identifier to the first entity;

receiving an indication of presence of a second entity within a FOD of a second sensor of the plurality of sensors at a sensor-relative position within a sensor-relative coordinate system of the second sensor;

translating the sensor-relative position of the second entity to an environment-relative position of the second entity within the environment-relative coordinate system; and

based on determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity, assigning the unique entity identifier to the second entity.

19. An entity-tracking computing system, comprising:

a logic machine;

a communication subsystem; and

a storage machine holding instructions executable by the logic machine to:

via the communication subsystem, communicatively couple the entity-tracking computing system to a plurality of sensors each having a field-of-detection (FOD) within an environment, the plurality of sensors including at least an infrared (IR) camera and a radar sensor;

maintain an environment-relative coordinate system to which the FOD of each sensor of the plurality of sensors is mapped;

receive, via the communication subsystem and from the IR camera, an indication of presence of a first entity within a FOD of the IR camera at a sensor-relative position within a sensor-relative coordinate system of the IR camera;

translate the sensor-relative position of the first entity to an environment-relative position of the first entity within the environment-relative coordinate system;

identify the first entity as a previously-identified human, and assign a unique entity identifier to the first entity, the unique entity identifier being a previously-assigned entity identifier associated with the previously-identified human;

receive, from the radar sensor, an indication of presence of a second entity within a FOD of the radar sensor at a sensor-relative position within a sensor-relative coordinate system of the radar sensor;

translate the sensor-relative position of the second entity to an environment-relative position of the second entity within the environment-relative coordinate system; and

based on determining that the environment-relative position of the second entity is consistent with the environment-relative position of the first entity, assign the unique entity identifier to the second entity.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2017
From: PRADEEP, VIVEK; SALA, PABLO LUIS; WEISS, JOHN GUIDO ATKINS; LUTZ, MOSHE RANDALL
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 043348/0951 →
Continuity (3)
Provisional Application 62482165 · Apr 5, 2017
Provisional Application 62459020 · Feb 14, 2017
Related Publication 20180231653A1 · Aug 16, 2018
Cited By (6)
US 12,248,057 US 12,253,620 US 12,270,657 US 12,293,773 US 12,493,355 US 12,658,197