IP Library Granted Patent US 9,987,752
Granted Patent B2
US 9,987,752 · App. 15/179,851 · Granted Jun 5, 2018

Systems and methods for automatic detection of spills

Inventors: Dimitry Fisher (San Diego, CA); Cody Griffin (San Diego, CA); Micah Richert (San Diego, CA); Filip Piekniewski (San Diego, CA); Eugene Izhikevich (San Diego, CA); Jayram Moorkanikara Nageswaran (San Diego, CA); John Black (San Diego, CA)
Assignee: BRAIN CORPORATION
B25J9/1697G06K9/00671G06K9/4652G06K9/4661G06T7/0079H04N5/33G06T2207/10024G06T2207/10048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,987,752
App. No.
15/179,851
Granted
Jun 5, 2018
Kind
B2
Abstract

Systems and methods for automatic detection of spills are disclosed. In some exemplary implementations, a robot can have a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between locations. The robot can process the at least one image by segmentation. Once the spill has been identified, the robot can then generate an alert indicative at least in part of a recognition of the spill.

Claims (43)

1. A robot comprising:

an actuator configured to move the robot between a plurality of locations;

a spill detector comprising at least one optical imaging device configured to capture a plurality of images of a scene containing a spill while the robot moves between the plurality of locations; and

a processor apparatus configured to identify the spill in the plurality of images, and generate an alert indicative at least in part of a recognition of the spill;

wherein:

the processor is further configured to determine a confidence measure in the identification of the spill;

the generation of the alert indicative at least in part of the recognition of the spill is based on the confidence measure meeting or exceeding a predetermined threshold, the confidence measure being determined based at least on a quantity of images containing at least a portion of the identified spill relative to a quantity of images captured over a predetermined time interval; and

the generation of the alert comprises a presentation of a plurality of user-selectable options associated with actions of the robot with respect to the identified spill.

2. The robot of claim 1 , wherein the at least one optical imaging device comprises an infrared camera and the plurality of images comprises thermal images.

3. The robot of claim 1 , further comprising a temperature adjuster configured to change a temperature value of the scene containing the spill.

4. The robot of claim 3 , wherein the temperature adjuster comprises at least one of an exhaust and a fan.

5. The robot of claim 1 , wherein the robot further comprises a sensor configured to detect at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the scene.

6. The robot of claim 5 , wherein the confidence measure is based at least in part on information from both the sensor and the at least one image.

7. The robot of claim 1 , wherein the processor is further configured to generate a color image having a plurality of colors based at least in part on a plurality of thermal values of a segment of the plurality of images, and the plurality of colors are indicative at least in part of the spill.

8. The robot of claim 1 , wherein the robot further comprises a floor cleaning system.

9. The robot of claim 8 , wherein the processor is further configured to receive an action command in response to the alert and to turn off the floor cleaning system based at least in part on the received action command.

10. The robot of claim 1 , wherein the processor is further configured to utilize a learning-based visual classification on a library of spill images, and identify the spill in the at least one image based at least in part on the learning-based visual classification.

11. A method for detecting a spill comprising:

generating a first image of a first scene at a first location that contains a spill;

generating a second image of a second scene at a second location that contains no spills;

segmenting the first image to detect the spill from at least thermal values in a segment of the first image, the segmenting of the first image comprising identifying a difference between (i) a known thermal value associated with the second scene and (ii) the at least thermal values in the segment of the first image;

identifying the spill based on the difference; and

generating an alert indicative at least in part of the identification of the spill.

12. The method of claim 11 , further comprising adjusting a temperature value of the first scene while generating the first image.

13. The method of claim 11 , further comprising determining a confidence value in the identified spill, wherein the generated alert is further indicative of the confidence value.

14. The method of claim 11 , further comprising sensing at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the first scene.

15. The method of claim 14 , further comprising determining a confidence value in the identified spill based at least in part on the segmentation of the first image and the sensed at least one of reflectance properties, emission properties, electrical properties, noises, and friction of the first scene, wherein the generated alert is further indicative of the confidence value.

16. The method of claim 11 , further comprising receiving an action command in response to the generated alert and performing an action in response to the action command.

17. A robot comprising:

an actuator configured to move the robot between a plurality of locations;

a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between the plurality of locations; and

a processor apparatus configured to:

generate an action command based at least in part on the at least one image;

receive feedback from an operator, the feedback comprising at least a confirmation of the generated action command; and

based at least on the confirmation:

perform an autonomous physical action with respect to the spill;

adjust a confidence parameter associated with a detection of the spill by the robot; and

associate the autonomous physical action performed by the robot with detection of a subsequent spill.

18. The robot of claim 17 , further comprising a temperature adjuster configured to change a temperature value of the scene containing the spill.

19. The robot of claim 17 , wherein the action command is a stop command that is configured to stop the robot from moving between the plurality of locations.

20. The robot of claim 17 , wherein:

the robot is in data communication with a user interface configured to receive the feedback;

the action command comprises a query by the robot, the query being related to at least one of (i) the detection of the spill and (ii) the autonomous physical action with respect to the spill.

Assignments (2)
SECURITY INTEREST Recorded Oct 8, 2021
From: BRAIN CORPORATION
To: HERCULES CAPITAL, INC.
Reel/Frame 057851/0574 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2016
From: FISHER, DIMITRY; GRIFFIN, CODY; RICHERT, MICAH; PIEKNIEWSKI, FILIP; IZHIKEVICH, EUGENE; NAGESWARAN, JAYRAM MOORKANIKARA; BLACK, JOHN
To: BRAIN CORPORATION
Reel/Frame 040061/0799 →
Continuity (1)
Related Publication 20170355081A1 · Dec 14, 2017