IP Library Granted Patent US 10,464,213
Granted Patent B2
US 10,464,213 · App. 15/997,397 · Granted Nov 5, 2019

Systems and methods for automatic detection of spills

Inventors: Dimitry Fisher (San Diego, CA); Cody Griffin (San Diego, CA); Micah Richert (San Diego, CA); Filip Piekniewski (San Diego, CA); Eugene Izhikevich (San Diego, CA); Jayram Moorkanikara Nageswaran (San Diego, CA); John Black (San Diego, CA)
Assignee: Brain Corporation
B25J9/1697G05D1/0016G06K9/00671G06K9/4652G06K9/4661G06T7/11G08B21/20H04N5/33G05D2201/0203G06K9/00771G06T2207/10024G06T2207/10048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,464,213
App. No.
15/997,397
Granted
Nov 5, 2019
Kind
B2
Abstract

Systems and methods for automatic detection of spills are disclosed. In some exemplary implementations, a robot can have a spill detector comprising at least one optical imaging device configured to capture at least one image of a scene containing a spill while the robot moves between locations. The robot can process the at least one image by segmentation. Once the spill has been identified, the robot can then generate an alert indicative at least in part of a recognition of the spill.

Claims (52)

1. A robot for detecting at least one item, comprising:

a memory having computer readable instructions stored thereon; and

at least one processor configured to execute the computer readable instructions to,

identify the at least one item in a plurality of images captured by the robot between a plurality of locations;

determine a confidence measure in the identification of the at least one item, the confidence measure being determined based on at least a quantity of images containing at least a portion of the identified at least one item relative to a quantity of images captured over a predetermined time interval;

recognize the at least one item based on the confidence measure meeting or exceeding a predetermined threshold; and

generate an alert notification in response to the recognizing of the at least one item, the alert notification includes plurality of user-selectable options corresponding to actions of the robot with respect to the at least one item.

2. The robot of claim 1 , further comprising:

at least one optical imaging device including an infrared camera, and

wherein,

the plurality of images includes thermal images, and

the at least one item corresponds to a spill on a floor.

3. The robot of claim 1 , further comprising:

a temperature adjuster configured to change a temperature value of a scene containing the at least one item, the temperature adjuster includes at least one of an exhaust and a fan.

4. The robot of claim 1 , further comprising:

a sensor configured to detect at least one of a reflectance property of a scene, emission property of the scene, electrical property of the scene, and friction property of the scene.

5. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,

receive an action command in response to the alert notification; and

transmit a signal to turn off a floor cleaning system based on the receipt of the action command.

6. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,

utilize a learning-based visual classification on a library of images corresponding to the at least one item; and

identify the at least one item based on the learning-based visual classification.

7. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,

generate a color image based on the at least thermal values in the segment of the first image; and

determine whether colors in the color image correspond to the at least one item.

8. A method for detecting at least one item, comprising:

segmenting a first image of a first scene to detect the at least one item from at least thermal values in a segment of the first image, the segmenting of the first image including identifying a difference between: (i) a known thermal value associated with a different second scene, and (ii) the at least thermal values in the segment of the first image;

identifying the at least one item based on the difference; and

generating an alert notification corresponding to the identification of the at least one item.

9. The method of claim 8 , further comprising:

adjusting a temperature value of the first scene while generating the first image.

10. The method of claim 8 , further comprising:

determining a confidence value in the identified at least one item; and

sensing at least one of reflectance properties of the first scene, emission properties of the first scene, electrical properties of the first scene, and friction properties of the first scene,

wherein the alert notification is indicative of the confidence value.

11. The method of claim 10 , wherein the confidence value is determined based at least in part on the segmentation of the first image and detection of at least one of the reflectance properties of the first scene, the emission properties of the first scene, the electrical properties of the first scene, and friction properties of the first scene.

12. The method of claim 8 , further comprising:

receiving an action command in response to the generated alert and performing an action in response to the action command.

13. The method of claim 8 , further comprising:

generating a color image based on the at least thermal values in the segment of the first image; and

determining whether colors in the color image correspond to the at least one item.

14. A non-transitory computer readable medium having computer readable instructions stored thereon that when executed by at least one processor configure the at least one processor to,

capture at least one image of a scene containing at least one item while a robot moves between a plurality of locations; and

generate an action command based on at least in part on the at least one image;

receive feedback from an operator, the feedback comprising at least a confirmation of the generated action command; and

transmit a signal, based on the at least one confirmation, to do at least one of the following: (i) perform an autonomous physical action with respect to the at least one item, (ii) adjust a confidence parameter associated with a detection of the at least one item, and (iii) associate the autonomous physical action performed by the robot with detection of a subsequent at least one item.

15. The non-transitory computer readable medium of claim 14 , wherein the at least one processor is further configured to execute the computer readable instructions to change a temperature value of the scene including the at least one item.

16. The non-transitory computer readable medium of claim 14 , wherein the non-transitory computer-readable medium coupled to a robot, the robot is in data communication with a user interface to receive feedback.

17. The non-transitory computer readable medium of claim 16 , wherein the robot is configured to receive at least one action command, via the user interface, from a user.

18. The non-transitory computer readable medium of claim 14 , wherein the at least one processor is further configured to,

generate a color image based on the at least thermal values in the segment of the first image; and

determine whether colors in the color image correspond to the at least one item.

Assignments (1)
SECURITY INTEREST Recorded Oct 8, 2021
From: BRAIN CORPORATION
To: HERCULES CAPITAL, INC.
Reel/Frame 057851/0574 →
Continuity (2)
Continuation 15179851 · Jun 10, 2016
Related Publication 20190061160A1 · Feb 28, 2019