IP Library › Granted Patent US 10,195,741
Granted Patent B2
US 10,195,741 · App. 15/224,109 · Granted Feb 5, 2019

Controlling a robot in the presence of a moving object

Inventors: Harshavardhana Kikkeri (Newcastle, WA); Bryan J. Thibodeau (Sammamish, WA)
Assignee: Microsoft Technology Licensing, LLC
B25J9/1676F16P3/142G06T1/0014G06T7/0004G06T7/20G05B2219/37567G05B2219/37571G05B2219/39091G05B2219/40202G05B2219/40203G05B2219/49152G05B2219/49158G06T2207/10021G06T2207/10028G06T2207/20036G06T2207/30196H04N13/204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,195,741
App. No.
15/224,109
Filed
Jul 29, 2016
Granted
Feb 5, 2019
Kind
B2
Art Unit
3664
USPC
700/259
Abstract

A method, system, and one or more computer-readable storage media for controlling a robot in the presence of a moving object are provided herein. The method includes capturing a number of frames from a three-dimensional camera system and analyzing a frame to identify a connected object. The frame is compared to a previous frame to identify a moving connected object (MCO). If an unexpected MCO is in the frame a determination is made if the unexpected MCO is in an actionable region. If so, the robot is instructed to take an action.

Claims (40)

1. A method, performed by a processor, for controlling a robot in the presence of a moving object, comprising:

analyzing a frame captured by a three-dimensional camera to identify a connected object comprising substantially proximate voxels;

determining motion vectors for the substantially proximate voxels based at least in part on the frame and a previous frame captured by the three-dimensional camera;

identifying a moving connected object (MCO) comprising the substantially proximate voxels, based at least in part on whether the motion vectors for the substantially proximate voxels are consistent;

determining if the MCO is an unexpected MCO based at least in part on an expected motion of the robot; and

instructing the robot to perform an action in response to determining the MCO is unexpected.

2. The method of claim 1 , wherein determining if the MCO is unexpected is based at least in part on auxiliary objects within the frame.

3. The method of claim 1 , wherein analyzing the frame comprises forming a group comprising the substantially proximate voxels to form connected objects.

4. The method of claim 1 , comprising determining if the unexpected MCO is in an actionable region.

5. The method of claim 4 , wherein the action taken is to activate an audible warning if the unexpected MCO is in the actionable region.

6. The method of claim 4 , wherein the action taken is to activate a visual warning if the unexpected MCO is in the actionable region.

7. The method of claim 4 , comprising slowing a motion of the robot if the unexpected MCO is in a warning zone.

8. The method of claim 4 , comprising stopping a motion of the robot if the unexpected MCO is in a hazard zone.

9. The method of claim 8 , comprising slowing a motion of the robot and moving the robot towards the unexpected MCO to allow interaction with the unexpected MCO.

10. The method of claim 2 , wherein identifying the MCO comprises:

comparing a location of a group of voxels forming a connected object to a previous location of the group of voxels forming the connected object;

eliminating any voxels that have inconsistent motion vectors; and

defining remaining voxels in the group of voxels as the MCO.

11. A system for controlling a robot in a presence of a moving object, comprising:

a three dimensional camera;

a processor; and

a system memory comprising code that, based at least in part on execution by the processor, is configured to:

analyze a frame captured by a three-dimensional camera to identify an object in relative motion to the robot;

determine motion vectors for voxels representing the object based at least in part on the frame and a previous frame captured by the three-dimensional camera;

group proximate voxels having consistent motion vectors;

determine if the object is unexpected based at least in part on an evaluation of expected relative motion of the robot and the grouped proximate voxels; and

instruct the robot to take an action if the object is determined to be unexpected.

12. The system of claim 11 , the robot being in motion.

13. The system of claim 11 , the object being in motion.

14. The system of claim 11 , the object being a moving connected object.

15. The system of claim 11 , the three-dimensional camera being part of the robot.

16. The system of claim 11 , wherein the system memory comprises code that, based at least in part on execution by the processor, is configured to activate a visual warning system, an audible warning system, or both, when the object enters a hazardous region or a warning region.

17. One or more computer-readable storage media for storing computer-readable instructions, the computer-readable instructions providing a system for controlling a robot in a presence of a moving object, the computer-readable instructions, based at least in part on execution by a processor, being configured to:

capture a plurality of frames from a three-dimensional camera system;

analyze one of the frames to identify a plurality of connected objects, each of the connected objects comprising substantially proximate voxels;

compare a location of each of the connected objects in the one frame to a location of each of the connected objects in a previous frame of the frames to determine motion vectors for the substantially proximate voxels;

compare the one frame to the previous frame to identify a moving connected object (MCO) comprising the substantially proximate voxels, wherein the motion vectors for the substantially proximate voxels are consistent;

determine if the MCO is an unexpected MCO based at least in part on an expected motion of the robot; and

instruct the robot to take an action.

18. The one or more computer-readable storage media of claim 17 , wherein the computer-readable instructions, based at least in part on execution by the processor, are configured to stop a motion of the robot if the unexpected MCO is in a hazardous region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2016
From: KIKKERI, HARSHAVARDHANA; THIBODEAU, BRYAN J.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039295/0161 →
Continuity (2)
Continuation 14172230 · Feb 4, 2014
Related Publication 20160354927A1 · Dec 8, 2016