IP Library › Granted Patent US 10,762,641
Granted Patent B2
US 10,762,641 · App. 16/195,955 · Granted Sep 1, 2020

Interaction recognition and analysis system

Inventors: Joshua George Abdoo (Stevensville, MI); Matthew P. Ebrom (Holland, MI); Neomar Giacomini (St. Joseph, MI); Donald John Gilmore (Berrien Springs, MI); Brian N. Radford (Stevensville, MI); Aaron Edward Showers (St. Joseph, MI); Collin Andrew Stipe (West Unity, OH)
Assignee: Whirlpool Corporation
G06T7/246G06F3/167G06K9/00G06K9/00362G06K9/00771
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,762,641
App. No.
16/195,955
Granted
Sep 1, 2020
Kind
B2
Abstract

An object tracking system is disclosed. The system comprises an appliance comprising at least one operating volume and an access region through which the operating volume is accessed. The system further comprises at least one imager and a controller. The controller is operable to detect at least one part of a person's body in the access region based on the image data and identify the object in connection with the part of the person's body. Based on the detection of the object in connection with the person's body, the controller is configured to identify the object. An action is determined for the object based on one or more characteristics of the physical object indicated in the image data.

Claims (40)

1. An object tracking system comprising:

an appliance comprising at least one operating volume and an access region through which the operating volume is accessed;

at least one imager configured to capture image data in a field of view comprising the access region; and

a controller in communication with the imager, wherein the controller is operable to:

identify a physical object in connection with the part of the person's body;

identify whether or not the physical object is in connection with at least one part of a person's body as the part of the person's body enters or exits the access region based on the image data; and

based on the identification of the physical object in connection with the part of the person's body, determine an action for the physical object based on one or more characteristics of the physical object indicated in the image data.

2. The system according to claim 1 , wherein the action is determined by the controller in response to the part of the person's body entering or exiting the operating volume in connection with the physical object based on the image data captured in the access region.

3. The system according to claim 1 , wherein the controller is operable to initiate the action to provide feedback to a user relating to the physical object.

4. The system according to claim 3 , wherein the controller is operable to determine an input to the controller from the user in response to the action.

5. The system according to claim 4 , wherein input from the user comprises a vocal communication or a gesture.

6. The system according to claim 1 , wherein the action comprises voice feedback to a user relating to the physical object.

7. The system according to claim 1 , wherein the action comprises selective lighting of at least a portion of a shelf or drawer to provide feedback to a user relating to the physical object.

8. The system according to claim 1 , wherein the operating volume comprises a processing region of the appliance in which the physical object is processed, stored, or consumed.

9. The system according to claim 1 , wherein the controller is configured to determine the action for the physical object in response to a detection of the physical object entering or exiting the operating volume.

10. A method of tracking an object utilized in connection with an appliance comprising the steps of:

capturing image data in an access region adjacent to an operating volume of the appliance;

detecting at least one part of a person's body in the access region of the operating volume based on the image data;

determining that the object is in connection with the part of the person's body;

determining that the object in connection with the part of the person's body is being inserted or withdrawn from the operating volume;

determining an action for the object in connection with part of the person's body based on one or more characteristics of the object indicated in the image data; and

in response to determining an action for the object, initiating the action to provide feedback to a user relating to the object in the access region.

11. The method according to claim 10 , wherein the action comprises a voice message or selective lighting.

12. The method according to claim 11 , wherein the voice message further comprises an indication of an optimal storage location for the object.

13. The method according to claim 10 , wherein the method further comprises determining an input to a controller in communication with an imager from the person's body in response to the action.

14. The method according to claim 13 , wherein determining an input from the person's body further comprises determining a gesture.

15. The method according to claim 13 , wherein determining an input from the person's body further comprises determining a voice response.

16. An object tracking system comprising:

an appliance comprising at least one operating volume and an access region through which the operating volume is accessed;

at least one imager configured to capture image data in a field of view comprising the access region; and

a controller in communication with the imager, wherein the controller is operable to:

detect at least one part of a person's body entering or exiting the access region based on the image data;

identify an object based on the image data;

identify that the object is in connection with the part of the person's body upon entering or exiting the operating volume based on the image data;

determine that the object in connection with the part of the person's body is being inserted or withdrawn from the operating volume; and

initiate an advanced interaction for the object in response to the entering or exiting based on one or more characteristics of the object indicated in the image data wherein the advanced interaction provides feedback to a user relating to the object.

17. The system according to claim 16 , wherein the appliance corresponds to a food storage appliance and the advanced interaction comprises selective lighting of at least a portion of the appliance to indicate an optimal storage location for the object.

18. The system according to claim 16 , wherein the appliance corresponds to a food storage appliance and the advanced interaction comprises a voice message inquiring about information relating to the object.

19. The system according to claim 18 , wherein the controller is further operable to determine an input to the controller from the user in response to the voice message.

20. The system according to claim 19 , wherein the input from the user comprises a gesture or a vocal communication.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2018
From: ABDOO, JOSHUA GEORGE; EBROM, MATTHEW P.; GIACOMINI, NEOMAR; GILMORE, DONALD JOHN; RADFORD, BRIAN N.; SHOWERS, AARON EDWARD; STIPE, COLLIN ANDREW
To: WHIRLPOOL CORPORATION
Reel/Frame 047550/0282 →
Continuity (2)
Continuation In Part 15364771 · Nov 30, 2016
Related Publication 20190087966A1 · Mar 21, 2019