IP Library Granted Patent US 10,467,510
Granted Patent B2
US 10,467,510 · App. 15/636,559 · Granted Nov 5, 2019

Intelligent assistant

Inventors: Haithem Albadawi (Redmond, WA); Zongyi Liu (Issaquah, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06K9/726A61B5/0205A61B5/0507A61B5/117A61B5/1113A61B5/7475G01S5/18G01S5/28G01S13/726G06F1/324G06F1/3206G06F1/3231G06F3/011G06F3/017G06F3/0304G06F3/0482G06F3/04842G06F3/167G06F17/271G06F17/279G06F21/32G06F21/35G06K9/00214G06K9/00255G06K9/00261G06K9/00288G06K9/00295G06K9/00342G06K9/00362G06K9/00711G06K9/00973G06K9/6254G06K9/6255G06K9/6289G06K9/6296G06N5/025G06N5/047G06N20/00G06T7/248G06T7/292G06T7/60G06T7/70G06T7/74G07C9/00111G08B13/1427G10L15/02G10L15/063G10L15/08G10L15/18G10L15/1815G10L15/1822G10L15/19G10L15/22G10L15/24G10L15/26G10L15/28G10L15/32G10L17/04G10L17/08G10L25/51H04L51/02H04L63/102H04L67/12H04L67/22H04N5/23219H04N5/332H04N7/181H04N7/188H04N21/231H04N21/42203H04N21/44218H04N21/44222H04R1/406H04R3/005H04W4/029H04W4/33A61B5/05A61B5/1118G01S5/16G01S11/14G01S13/38G01S13/867G01S13/888G06F3/0488G06F16/70G06F2203/0381G06F2221/2111G06K2209/09G06N3/0445G06T2207/10016G06T2207/10024G06T2207/20101G06T2207/30196G06T2207/30201G06T2207/30204G06T2207/30232G07C9/00134G08B29/186G10L17/00G10L2015/0635G10L2015/088G10L2015/223G10L2015/225G10L2015/228H04N5/247Y02D10/126Y02D10/173
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,467,510
App. No.
15/636,559
Filed
Jun 28, 2017
Granted
Nov 5, 2019
Kind
B2
Art Unit
2667
USPC
382/118
Abstract

Examples are disclosed herein that relate to entity tracking. One examples provides a computing device comprising a logic processor and a storage device holding instructions executable by the logic processor to receive image data of an environment including a person, process the image data using a face detection algorithm to produce a first face detection output at a first frequency, determine an identity of the person based on the first face detection output, and process the image data using another algorithm that uses less computational resources of the computing device than the face detection algorithm. The instructions are further executable to track the person within the environment based on the tracking output, and perform one or more of updating the other algorithm using a second face detection output, and updating the face detection algorithm using the tracking output.

Claims (43)

1. A computing device, comprising:

a logic processor; and

a storage device holding instructions executable by the logic processor to:

receive image data of an environment including a person;

process the image data using a face detection algorithm to produce a first face detection output at a first frequency;

determine an identity of the person based on the first face detection output;

process the image data using another algorithm that uses less computational resources of the computing device than the face detection algorithm, the other algorithm producing a tracking output at a second frequency greater than the first frequency;

track the person within the environment based on the tracking output; and

perform one or more of updating the other algorithm using a second face detection output, and updating the face detection algorithm using the tracking output.

2. The computing device of claim 1 , wherein the instructions are executable to process the image data using the other algorithm while not processing the image data using the face detection algorithm.

3. The computing device of claim 1 , wherein the instructions are executable to select the other algorithm based on a battery life condition associated with the computing device.

4. The computing device of claim 1 , wherein the instructions are executable to select the other algorithm based on an allocation of resources of the logic processor to one or more tasks other than processing of the image data.

5. The computing device of claim 1 , wherein the other algorithm is a color tracking algorithm, and wherein the tracking output includes a color histogram.

6. The computing device of claim 1 , wherein the other algorithm is a motion tracking algorithm, and wherein the tracking output includes one or more motion vectors.

7. The computing device of claim 1 , wherein the other algorithm is a first tracking algorithm, and wherein the instructions are executable to process the image data using a second tracking algorithm that uses less computational resources of the computing device than the face detection algorithm, wherein the tracking output of the first tracking algorithm is provided as input to the second tracking algorithm.

8. The computing device of claim 1 , wherein the instructions executable to update the face detection algorithm using the tracking output are executable to update a target region of the image data in which to prioritize the processing of the image data using the face detection algorithm, the target region corresponding to a location of the person tracked by the tracking output.

9. The computing device of claim 1 , wherein the instructions executable to update the other algorithm using the second face detection output are executable to update an association between a region of the image data in which a face of the person is detected and the tracking output.

10. The computing device of claim 1 , further comprising one or more image sensors integrated in the computing device and being configured to collect the image data, and wherein the computing device is an all-in-one computing device located in the environment.

11. The computing device of claim 1 , wherein the image data is received from one or more image sensors remotely located from the computing device.

12. A method of entity tracking at a computing device, comprising:

receiving image data of an environment including a person;

processing the image data using a face detection algorithm to produce a first face detection output at a first frequency;

determining an identity of the person based on the first face detection output;

processing the image data using another algorithm that uses less computational resources of the computing device than the face detection algorithm, the other algorithm producing a tracking output at a second frequency greater than the first frequency;

tracking the person within the environment based on the tracking output; and

performing one or more of updating the other algorithm using a second face detection output, and updating the face detection algorithm using the tracking output.

13. The method of claim 12 , wherein the other algorithm is a body shape tracking algorithm, and wherein the tracking output includes a body model.

14. The method of claim 12 , wherein the other algorithm is a gait tracking algorithm, and wherein the tracking output includes gait analysis data.

15. The method of claim 12 , wherein updating the face detection algorithm using the tracking output includes updating a target region of the image data in which to prioritize the processing of the image data using the face detection algorithm, the target region corresponding to a location of the person tracked by the tracking output.

16. The method of claim 12 , wherein updating the other algorithm using the second face detection output includes updating an association between a region of the image data in which a face of the person is detected and the tracking output.

17. The method of claim 12 , wherein the other algorithm is selected from among two or more tracking algorithms configured to produce respective tracking outputs each having a corresponding datatype.

18. The method of claim 12 , wherein the other algorithm is a first tracking algorithm, the method further comprising processing the image data using a second tracking algorithm that uses less computational resources of the computing device than the face detection algorithm, wherein the tracking output of the first tracking algorithm is provided as input to the second tracking algorithm.

19. An all-in-one computing device, comprising:

one or more image sensors;

a logic processor; and

a storage device holding instructions executable by the logic processor to:

receive image data of an environment including a person from the one or more image sensors;

process the image data using a face detection algorithm to produce a first face detection output at a first frequency;

determine an identity of the person based on the first face detection output;

process the image data using another algorithm that uses less computational resources of the computing device than the face detection algorithm, the other algorithm producing a tracking output at a second frequency greater than the first frequency;

track the person within the environment based on the tracking output; and

perform one or more of updating the other algorithm using a second face detection output, and updating the face detection algorithm using the tracking output.

20. The computing device of claim 19 , wherein the other algorithm is a first tracking algorithm, and wherein the instructions are executable to process the image data using a second tracking algorithm that uses less computational resources of the computing device than the face detection algorithm, wherein the tracking output of the first tracking algorithm is provided as input to the second tracking algorithm.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2017
From: ALBADAWI, HAITHEM; LIU, ZONGYI
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 042855/0045 →
Continuity (3)
Provisional Application 62459020 · Feb 14, 2017
Provisional Application 62482165 · Apr 5, 2017
Related Publication 20180232563A1 · Aug 16, 2018
Cited By (2)
US 12,253,620 US 12,493,355