Granted Patent B2

US 12,629,822 · App. 17/935,879 · Granted May 19, 2026

Inventors: Andras Gabor Kupcsik (Boeblingen, DE); Meng Guo (Beijing, CN)

Assignee: ROBERT BOSCH GMBH

B25J9/163B25J9/161B25J9/1697B25J19/023

Abstract

Claims (48)

1 . A method for controlling a robot, comprising:

Performing demonstrations, whereineach demonstration of the demonstrations demonstrates a handling of an object;

Providing, for each demonstration, at least one descriptor image of the object from a point of view of the robot, wherein the descriptor image specifies feature points for locations on the object;

Selecting a set of feature points from the specified feature points, wherein the feature points are selected by searching an optimum of an objective function which rewards selected feature points being visible in the descriptor images;

Training a robot control modelusing the demonstrations, wherein the robot control model is configured to output control information depending on an input object pose; and

Controlling the robot for a control scene with the object by:

Determining a descriptor image of the object from the point of view of the robot;

Locating the selected set of feature points in the descriptor image of the object;

Determining Euclidean coordinates of the located feature points for the control scene;

Estimating a pose from the determined Euclidean coordinates; and

Controlling the robot to handle the object using the robot control model, wherein the estimated pose is supplied to the robot control model as input.

2 . The method of claim 1 , wherein the objective function further rewards one or more of the selected feature points being spaced apart in descriptor space, locations on the object corresponding to the selected feature points being spaced apart in Euclidean space, and a detection error for the selected feature points for the object being low.

3 . The method of claim 1 , further comprising:

Matching a plane to the object;

Selecting the feature points such that they define a coordinate frame on the plane; and

Estimating the pose from the determined Euclidean coordinates of the located feature points and information about a pose of the matched plane.

4 . The method of claim 3 , wherein estimating the pose from the determined Euclidean coordinates includes projecting the Euclidean coordinates of the located feature points to the matched plane.

5 . The method of claim 3 , wherein the feature points are selected such that they define a coordinate frame on the plane and the pose is estimated from the determined Euclidean coordinates of the located feature points and information about the pose of the plane when a variation of the object in a spatial direction is below a predetermined threshold.

6 . The method of claim 1 , further comprising:

Determininga derivation rule of a coordinate frame from Euclidean coordinates of the selected feature points;

Wherein estimating the pose from the determined Euclidean coordinates includes application of the derivation rule to the selected feature points, wherein the derivation rule is determined by searching a minimum of a dependency of the coordinate frame from noise in the Euclidean coordinates.

7 . The method of claim 1 , wherein the training of the robot control model using the demonstrations includes, for each demonstration, locating the selected set of feature points in the descriptor image of the object, determining Euclidean coordinates of the located feature points for the demonstration, and estimating a pose from the determined Euclidean coordinates for the demonstration.

8 . The method of claim 1 , further comprising determining the descriptor image of the object from a camera image of the object by a dense object net.

9 . The method of claim 1 , wherein the estimated pose is invariant with respect to a movement of the object such that when the object is moved the estimated pose is transformed in the same way.

10 . A robot controller, configured to control a robot, the robot controller configured to:

Perform demonstrations, wherein each demonstration of the demonstrations demonstrates a handling of an object;

Provide, for each demonstration, at least one descriptor image of the object from a point of view of the robot, wherein the descriptor image specifies feature points for locations on the object;

Select a set of feature points from the specified feature points, wherein the feature points are selected by searching an optimum of an objective function which rewards selected feature points being visible in the descriptor images;

Train a robot control model using the demonstrations, wherein the robot control model is configured to output control information depending on an input object pose; and

Control the robot for a control scene with the object by:

Determining a descriptor image of the object from the point of view of the robot;

Locating the selected set of feature points in the descriptor image of the object;

Determining Euclidean coordinates of the located feature points for the control scene;

Estimating a pose from the determined Euclidean coordinates; and

Controlling the robot to handle the object using the robot control model, wherein the estimated pose is supplied to the robot control model as input.

11 . The robot controller of claim 10 , wherein the estimated pose is invariant with respect to a movement of the object such that when the object is moved the estimated pose is transformed in the same way.

12 . A non-transitory computer-readable medium on which is stored a computer program for controlling a robot, the control program, when executed by a computer, causing the computer to perform:

Performing demonstrations, wherein each demonstration of the demonstrations demonstrates a handling of an object;

Providing, for each demonstration, at least one descriptor image of the object from a point of view of the robot, wherein the descriptor image specifies feature points for locations on the object;

Selecting a set of feature points from the specified feature points, wherein the feature points are selected by searching an optimum of an objective function which rewards selected feature points being visible in the descriptor images;

Training a robot control modelusing the demonstrations, wherein the robot control model is configured to output control information depending on an input object pose; and

Controlling the robot for a control scene with the object by:

Determining a descriptor image of the object from the point of view of the robot;

Locating the selected set of feature points in the descriptor image of the object;

Determining Euclidean coordinates of the located feature points for the control scene;

Estimating a pose from the determined Euclidean coordinates; and

Controlling the robot to handle the object using the robot control model, wherein the estimated pose is supplied to the robot control model as input.

13 . The non-transitory computer-readable medium of claim 12 , wherein the estimated pose is invariant with respect to a movement of the object such that when the object is moved the estimated pose is transformed in the same way.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (15)