IP Library Granted Patent US 9,740,193
Granted Patent B2
US 9,740,193 · App. 14/432,732 · Granted Aug 22, 2017

Sensor-based safety features for robotic equipment

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,740,193
App. No.
14/432,732
Granted
Aug 22, 2017
Kind
B2
Abstract

Technologies are generally described for sensor-based safety features for robotic equipment, and the implementation thereof. One or more sensors may be positioned relative to the robotic equipment such that the sensors may capture light from at least a portion of an environment surrounding the robotic equipment. In some examples, the sensors may be integrated with the robotic equipment and/or may be configured to rotate. An analysis module coupled to the sensors may build a model image of the environment based on the light captured by the sensors. The analysis module may detect that an unintended object is approaching the robotic equipment in response to detecting a change in the model image, and based on a proximity and/or a speed of approach of the object to the robotic equipment, the analysis module may instruct the robotic equipment to reduce an operating speed and/or stop motion of the robotic equipment.

Claims (52)

1. A method to implement sensor-based safety features for robotic equipment, the method comprising:

generating a model image of an environment that surrounds the robotic equipment based on light captured by one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

receiving, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

detecting a change in the model image based on a comparison of one or more of the plurality of frames;

determining whether an unintended object is approaching the robotic equipment based on the detected change;

in response to a determination that the unintended object is approaching the robotic equipment, instructing the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment; and

in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment, instructing the robotic equipment to increase the operating speed of the robotic equipment,

wherein the detection that the unintended object is leaving the second boundary comprises storing the plurality of frames as histograms and determining an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

2. The method of claim 1 , further comprising:

determining the one or more of the proximity and the speed of approach of the unintended object to the robotic equipment based on the comparison of the one or more of the plurality of frames.

3. The method of claim 1 , wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment to substantially zero.

4. The method of claim 1 , wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to stop motion in response to detection that the unintended object is at a location between the first boundary and the robotic equipment.

5. The method of claim 1 , wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment in response to detection that the unintended object is at a location between the second boundary and the first boundary.

6. The method of claim 1 , further comprising instructing the robotic equipment to restart motion in response to detection that the unintended object is leaving the first boundary.

7. An apparatus configured to implement sensor-based safety features for robotic equipment, the apparatus comprising:

one or more sensors configured to capture light from an environment that surrounds the robotic equipment; and

at least one analysis module communicatively coupled to the one or more sensors, wherein the at least one analysis module is configured to:

generate a model image of the environment that surrounds the robotic equipment based on the light captured by the one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

receive, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

detect a change in the model image based on a comparison of one or more of the plurality of frames;

determine whether an unintended object is approaching the robotic equipment based on the detected change;

in response to a determination that the unintended object is approaching the robotic equipment, instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment; and

in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment, instruct the robotic equipment to increase the operating speed of the robotic equipment,

wherein to detect that the unintended object is leaving the second boundary, the at least one analysis module is configured to store the plurality of frames as histograms and to determine an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

8. The apparatus of claim 7 , wherein the one or more sensors comprise at least one of complementary metal oxide semiconductor (CMOS) image sensors, charged coupled device (CCD) image sensors, and N-type metal oxide semiconductor (NMOS) image sensors.

9. The apparatus of claim 7 , wherein the one or more sensors are positioned at one or more locations relative to the robotic equipment such that the light is captured from at least one portion of the environment that surrounds the robotic equipment.

10. The apparatus of claim 7 , wherein the one or more sensors are integrated with the robotic equipment.

11. The apparatus of claim 7 , wherein the one or more sensors are positioned or configured to rotate such that the light is captured from substantially an entirety of the environment that surrounds the robotic equipment.

12. The apparatus of claim 7 , wherein the at least one analysis module is configured to employ one or more Gaussian distributions to generate the model image.

13. The apparatus of claim 7 ,

wherein the user interface is coupled to the at least one analysis module and the user interface is configured to display the model image of the environment that surrounds the robotic equipment to an operator of the robotic equipment.

14. A system to implement sensor-based safety features for robotic equipment, the system comprising:

at least one imaging module configured to capture light of an environment that surrounds the robotic equipment through one or more sensors;

at least one modeling module configured to generate a model image of the environment that surrounds the robotic equipment based on the light captured by the at least one imaging module, wherein a time variation of the model image is stored as a plurality of frames;

at least one detection module configured to detect a change in the model image based on a comparison of one or more of the plurality of frames, and to determine whether an unintended object is approaching the robotic equipment based on the detected change; and

at least one controller configured to:

receive, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment, in response to a determination that the unintended object is approaching the robotic equipment; and

instruct the robotic equipment to increase the operating speed of the robotic equipment in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment,

wherein to detect that the unintended object is leaving the second boundary, the at least one detection module is configured to store the plurality of frames as histograms and to determine an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

15. The system of claim 14 , wherein the at least one modeling module is further configured to:

classify pixels of the model image into one of a foreground and a background of the model image;

group the pixels classified as the foreground of the model image; and

assign a threshold value for a size of the grouped pixels.

16. The system of claim 15 , wherein to detect the change in the model image, the at least one detection module is configured to:

determine that the size of the grouped pixels exceeds the threshold value.

Assignments (5)
LICENSE Recorded Mar 27, 2019
From: IDF HOLDINGS I LLC
To: ROBOTIC VISION TECH INC
Reel/Frame 048709/0479 →
CHANGE OF NAME Recorded Jun 7, 2017
From: ROBOTICVISIONTECH LLC
To: ROBOTICVISIONTECH, INC.
Reel/Frame 042713/0147 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME NEEDS TOBE CORRECTED FROM ROBOTIC VISION TECHNOLOGIES, LLCTO ROBOTICVISIONTECH LLC PREVIOUSLY RECORDED ON REEL 035304 FRAME 0365. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 7, 2017
From: EMPIRE TECHNOLOGY DEVELOPMENT LLC
To: ROBOTICVISIONTECH LLC
Reel/Frame 042715/0977 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2015
From: HUNT, SHAWN
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 035304/0300 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2015
From: EMPIRE TECHNOLOGY DEVELOPMENT LLC
To: ROBOTIC VISION TECHNOLOGIES, LLC
Reel/Frame 035304/0364 →