Robot teaching with scans in and out of robot workspace

1 . A robot teaching system, comprising:

a first 3D scanning device and a second 3D scanning device;

a user-input device;

a computation device; and

a storage device in communication with the computation device, the storage device storing instructions that when executed by the computation device results in a method comprising:

causing the first 3D scanning device to capture a first set of 3D information of a workpiece;

recording a first spatial point in response to user input via the user input device;

determining at least one best-fit geometry of at least a portion of the first set of 3D information;

projecting the first spatial point to a first geometric point having a predefined relationship to the at least one best-fit geometry;

display to the user the first geometric point in real time with the user input;

causing the second 3D scanning device to capture additional geometric information of a region of the workpiece to which the first spatial point had been proximal;

updating the first set of 3D information with the additional geometric information to create merged 3D information;

projecting the first spatial point to a second geometric point having a predefined relationship to an individual one of the best-fit geometries of the merged 3D information; and

defining a robot instruction based on the second geometric point.

2 . The system of claim 1 , wherein the method executed by the computation device further comprises causing the second 3D scanning device to capture a second set of 3D information about the workpiece, in addition to the first set of 3D information captured by the first 3D scanning device, using an electric or force based measurement sensor, when a portion of a robot tool comes into contact with the workpiece.

3 . The system of claim 1 , wherein the user-input device provides visual, audible, and/or haptic feedback to indicate status.

4 . The system of claim 1 , wherein the user-input device provides visual, audible, and/or haptic feedback indicative of whether or not a potential pose is reachable by the robot.

5 . The system of claim 1 , wherein the user-input device provides visual, audible, and/or haptic feedback when the user-input device is not visible to a first 3D scanning device.

6 . The system of claim 1 , wherein the user-input device provides visual, audible, and/or haptic feedback when the user-input device has lost communication with the computation device.

7 . The system of claim 1 , wherein an intended path of a tool of the robot is shown to the user at the at least one geometric points in real time with the user input.

8 . The system of claim 1 , wherein a representation of the robot or part of the robot is shown to the user at the at least one geometric points in real time with the user input.

9 . The system of claim 1 , wherein the user repositions the first 3D scanning device or the second 3D scanning device by causing the robot to follow the user-input device to get at least one more view of the workpiece.

10 . The system of claim 1 , wherein the user repositions the first 3D scanning device or the second 3D scanning device by jogging the robot to get at least one more view of the workpiece.

11 . The system of claim 1 , wherein the user repositions the first 3D scanning device or the second 3D scanning device by selecting a predefined viewing location to get at least one more view of the workpiece.

12 . The system of claim 1 , wherein the computation device will create additional robot instructions based on process-based rules.

13 . The system of claim 1 , wherein a scan is made after the robot instructions are executed and a comparison between work intended to be performed and actual work performed will be shown to the user.

14 . The system of claim 13 , wherein the comparison shown to the user has visually recognizable attributes defined by thresholds which determine if a difference is of sufficient size to be shown by the visually recognizable attributes.

15 . The system of claim 1 , wherein the method executed by the computation device further comprises collecting 3D information about the workpiece, in addition to the first set of 3D information captured by the first 3D scanning device, by bringing a portion of a robot tool in contact with the workpiece.

16 . A robot teaching system, comprising:

at least one 3D scanning device;

a user input device;

an external axis device having at least one degree of freedom (DOF) for altering a relative position or a relative orientation between a workpiece and the at least one 3D scanning device;

a computation device; and

a storage device in communication with the computation device, the storage device storing instructions executable by the computation device to:

cause the at least one 3D scanning device to capture a first 3D scan of the workpiece for one or more of the relative positions or relative orientations;

determine two or more best-fit geometries of the workpiece from the first 3D scan and creating a 3D representation of the workpiece from the two or more best-fit geometries, the two or more best-fit geometries including a plane or cylinder;

recording one or more spatial points in response to user input via the user input device;

project the one or more spatial points to one or more geometric points having a predefined relationship to the 3D representation of the workpiece created from the two or more best-fit geometries; and

define a robot instruction based on the one or more geometric points.

17 . A robot teaching system, comprising:

at least one 3D scanning device;

a user input device;

an external axis device having at least one degree of freedom (DOF);

a computation device; and

a storage device in communication with the computation device, the storage device storing instructions executable by the computation device to:

cause the at least one 3D scanning device to capture a first 3D scan of a workpiece;

determine two or more best-fit geometries of the workpiece from the first 3D scan and creating a 3D representation of the workpiece from the two or more best-fit geometries, the two or more best-fit geometries including a plane or cylinder;

recording one or more spatial points in response to user input via the user input device;

project the one or more spatial points to one or more geometric points having a predefined relationship to the 3D representation of the workpiece created from the two or more best-fit geometries;

define a robot instruction based on the one or more geometric points; and

altering a position or orientation of the workpiece via the external axis device.

18 . The system of claim 1 , wherein the method executed by the computation device further comprises using a machine learning model to predict a quality resulting from the robot instruction.

19 . The system of claim 1 , wherein the method executed by the computation device further comprises recording the results of the robot instruction to train a machine learning model.